2024-09-16T11:32:56.047 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T11:32:56.067 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T11:32:56.139 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907206 branch: wip-bharath9-testing-2024-09-13-1444-quincy description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/cache thrashosds-health workloads/rbd_fsx_rate_limit} email: sangadi@redhat.com first_in_suite: false job_id: '7907206' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath9-testing-2024-09-13-1444-quincy ceph: conf: client: rbd qos iops burst: 100 rbd qos iops limit: 50 rbd qos schedule tick min: 100 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(CACHE_POOL_NEAR_FULL\) - \(CACHE_POOL_NO_HIT_SET\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - mons down - mon down - out of quorum sha1: d75f23820806da622f561ee5ee7551015285993f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: enable_coprs: - ceph/el9 flavor: default sha1: d75f23820806da622f561ee5ee7551015285993f thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath9-testing-2024-09-13-1444-quincy sha1: d75f23820806da622f561ee5ee7551015285993f owner: scheduled_sangadi@teuthology priority: 130 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 9764 sha1: d75f23820806da622f561ee5ee7551015285993f sleep_before_teardown: 0 subset: 111/120000 suite: rbd suite_branch: wip-bharath9-testing-2024-09-13-1444-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d75f23820806da622f561ee5ee7551015285993f targets: smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPc++QQ1w+1BamHjutN/d3o3/GMdv+Qh4ZfhfnaWNiAGwRtQz0dM7dnkMQLsozwhCkp0Cd1JQ11z2aRmFeNM6bM= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCCNyl/oqltRRr7bcKNOoN8heGT9kiGQBHhiGOt499Hn8rod5PGOXPfDIqACF6JXQmU7sBL78Fk+49X2VBm6t0o= tasks: - install: null - ceph: null - exec: client.0: - sudo ceph osd pool create cache 4 - sudo ceph osd tier add rbd cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay rbd cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 60 - sudo ceph osd pool set cache target_max_objects 250 - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_10:44:09 tube: smithi user: sangadi verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T11:32:56.139 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa; will attempt to use it 2024-09-16T11:32:56.140 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa/tasks 2024-09-16T11:32:56.140 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T11:32:56.142 INFO:teuthology.task.internal:Checking packages... 2024-09-16T11:32:56.164 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd75f23820806da622f561ee5ee7551015285993f' 2024-09-16T11:32:56.164 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T11:32:56.164 INFO:teuthology.packaging:ref: None 2024-09-16T11:32:56.164 INFO:teuthology.packaging:tag: None 2024-09-16T11:32:56.164 INFO:teuthology.packaging:branch: wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-16T11:32:56.164 INFO:teuthology.packaging:sha1: d75f23820806da622f561ee5ee7551015285993f 2024-09-16T11:32:56.164 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-16T11:32:56.309 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1643.gd75f2382 2024-09-16T11:32:56.310 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T11:32:56.317 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T11:32:56.317 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T11:32:56.359 INFO:teuthology.task.internal:Saving configuration 2024-09-16T11:32:56.372 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T11:32:56.384 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T11:32:56.409 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907206', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 11:27:52.132528', 'locked_by': 'scheduled_sangadi@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPc++QQ1w+1BamHjutN/d3o3/GMdv+Qh4ZfhfnaWNiAGwRtQz0dM7dnkMQLsozwhCkp0Cd1JQ11z2aRmFeNM6bM='} 2024-09-16T11:32:56.430 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907206', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 11:27:52.130571', 'locked_by': 'scheduled_sangadi@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCCNyl/oqltRRr7bcKNOoN8heGT9kiGQBHhiGOt499Hn8rod5PGOXPfDIqACF6JXQmU7sBL78Fk+49X2VBm6t0o='} 2024-09-16T11:32:56.430 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T11:32:56.436 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-16T11:32:56.436 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-16T11:32:56.436 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T11:32:56.516 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb71326fac0>, signals=[15]) 2024-09-16T11:32:56.517 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T11:32:56.522 INFO:teuthology.task.internal:Opening connections... 2024-09-16T11:32:56.522 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T11:32:56.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:32:56.599 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-09-16T11:32:56.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:32:56.671 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T11:32:56.678 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2024-09-16T11:32:56.696 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2024-09-16T11:32:56.696 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-09-16T11:32:56.754 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-09-16T11:32:56.754 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2024-09-16T11:32:56.754 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-09-16T11:32:56.754 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-09-16T11:32:56.754 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2024-09-16T11:32:56.754 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2024-09-16T11:32:56.754 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T11:32:56.755 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-09-16T11:32:56.755 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2024-09-16T11:32:56.755 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T11:32:56.755 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-09-16T11:32:56.755 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T11:32:56.755 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T11:32:56.755 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T11:32:56.756 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-09-16T11:32:56.784 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-09-16T11:32:56.802 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-09-16T11:32:56.802 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-09-16T11:32:56.860 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-09-16T11:32:56.860 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-09-16T11:32:56.860 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-09-16T11:32:56.860 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-09-16T11:32:56.860 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-09-16T11:32:56.860 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-09-16T11:32:56.860 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T11:32:56.860 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-09-16T11:32:56.860 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-09-16T11:32:56.861 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T11:32:56.861 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-09-16T11:32:56.861 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T11:32:56.861 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T11:32:56.861 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T11:32:56.861 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-09-16T11:32:56.875 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T11:32:56.884 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T11:32:56.903 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T11:32:56.903 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2024-09-16T11:32:56.906 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-09-16T11:32:56.922 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T11:32:56.928 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T11:32:56.928 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2024-09-16T11:32:56.964 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-09-16T11:32:57.059 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T11:32:57.108 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T11:32:57.125 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T11:32:57.125 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T11:32:57.126 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T11:32:57.126 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T11:32:57.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:32:57.143 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-16T11:32:57.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:32:57.145 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-09-16T11:32:57.201 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-503.el9.x86_64 2024-09-16T11:32:57.201 INFO:teuthology.task.kernel:Running kernel on smithi070: 5.14.0-503.el9.x86_64 2024-09-16T11:32:57.201 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum install -y kernel 2024-09-16T11:32:57.202 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-503.el9.x86_64 2024-09-16T11:32:57.202 INFO:teuthology.task.kernel:Running kernel on smithi192: 5.14.0-503.el9.x86_64 2024-09-16T11:32:57.203 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum install -y kernel 2024-09-16T11:32:59.672 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-16T11:33:03.486 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.2 MB 00:04 2024-09-16T11:33:05.620 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-16T11:33:09.846 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-16T11:33:11.238 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 95 kB/s | 19 kB 00:00 2024-09-16T11:33:12.558 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-16T11:33:17.914 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T11:33:19.595 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 20 MB 00:18 2024-09-16T11:33:19.895 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T11:33:19.895 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T11:33:19.993 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:33:20.009 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:33:20.009 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-16T11:33:20.010 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:33:20.010 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:33:20.010 INFO:teuthology.orchestra.run.smithi070.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T11:33:20.010 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T11:33:20.010 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T11:33:20.010 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T11:33:20.010 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T11:33:20.010 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:33:20.010 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:33:20.010 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:33:20.010 INFO:teuthology.orchestra.run.smithi070.stdout:Install 4 Packages 2024-09-16T11:33:20.010 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:33:20.011 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 79 M 2024-09-16T11:33:20.011 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 126 M 2024-09-16T11:33:20.011 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:33:20.232 INFO:teuthology.orchestra.run.smithi070.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 272 kB/s | 23 kB 00:00 2024-09-16T11:33:20.857 INFO:teuthology.orchestra.run.smithi070.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-16T11:33:21.457 INFO:teuthology.orchestra.run.smithi070.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 26 MB/s | 35 MB 00:01 2024-09-16T11:33:21.849 INFO:teuthology.orchestra.run.smithi070.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 18 MB/s | 29 MB 00:01 2024-09-16T11:33:21.849 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:33:21.849 INFO:teuthology.orchestra.run.smithi070.stdout:Total 43 MB/s | 79 MB 00:01 2024-09-16T11:33:22.531 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:33:22.886 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:33:22.887 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:33:23.846 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:33:23.847 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:33:24.079 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-16T11:33:25.538 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 90 kB/s | 19 kB 00:00 2024-09-16T11:33:26.285 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:33:26.768 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-16T11:33:26.932 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T11:33:27.007 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T11:33:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T11:33:29.928 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T11:33:32.274 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-16T11:33:34.280 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T11:33:34.281 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T11:33:34.362 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:33:34.372 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:33:34.372 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-09-16T11:33:34.372 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:33:34.372 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:33:34.373 INFO:teuthology.orchestra.run.smithi192.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T11:33:34.373 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-16T11:33:34.373 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T11:33:34.373 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T11:33:34.373 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T11:33:34.373 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:33:34.373 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:33:34.373 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:33:34.373 INFO:teuthology.orchestra.run.smithi192.stdout:Install 4 Packages 2024-09-16T11:33:34.373 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:33:34.374 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 79 M 2024-09-16T11:33:34.374 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 126 M 2024-09-16T11:33:34.374 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:33:34.492 INFO:teuthology.orchestra.run.smithi192.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 234 kB/s | 23 kB 00:00 2024-09-16T11:33:34.730 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T11:33:34.769 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T11:33:34.917 INFO:teuthology.orchestra.run.smithi192.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-09-16T11:33:35.343 INFO:teuthology.orchestra.run.smithi192.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 36 MB/s | 35 MB 00:00 2024-09-16T11:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 26 MB/s | 29 MB 00:01 2024-09-16T11:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout:Total 64 MB/s | 79 MB 00:01 2024-09-16T11:33:36.286 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:33:36.635 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:33:36.635 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:33:37.590 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:33:37.590 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:33:39.600 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T11:33:40.058 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:33:40.732 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T11:33:40.803 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T11:33:43.607 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T11:33:43.740 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T11:33:48.580 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T11:33:48.699 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T11:33:53.645 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T11:34:17.690 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T11:34:17.732 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T11:34:17.999 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T11:34:17.999 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T11:34:18.000 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T11:34:18.000 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T11:34:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T11:34:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:34:18.793 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:34:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T11:34:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T11:34:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T11:34:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T11:34:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:34:18.794 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:34:19.158 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T11:34:19.243 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T11:34:19.243 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T11:34:19.243 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T11:34:19.243 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T11:34:19.244 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T11:34:19.244 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T11:34:19.244 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T11:34:19.244 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T11:34:19.303 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T11:34:19.303 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T11:34:19.303 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2024-09-16T11:34:19.342 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2024-09-16T11:34:19.342 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2024-09-16T11:34:19.342 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.06 2024-09-16T11:34:19.342 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 82.el9 2024-09-16T11:34:19.342 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2024-09-16T11:34:19.342 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T11:34:19.342 INFO:teuthology.orchestra.run.smithi070.stdout:Group : Unspecified 2024-09-16T11:34:19.342 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 8274670 2024-09-16T11:34:19.342 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2024-09-16T11:34:19.342 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T11:34:19.343 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T11:34:19.343 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T11:34:19.343 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T11:34:19.343 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : builder@centos.org 2024-09-16T11:34:19.343 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : CentOS 2024-09-16T11:34:19.343 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T11:34:19.343 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2024-09-16T11:34:19.343 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2024-09-16T11:34:19.343 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:34:19.343 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T11:34:19.343 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T11:34:19.344 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T11:34:19.344 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2024-09-16T11:34:19.344 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:34:19.344 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T11:34:19.345 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T11:34:19.345 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 5.14.0-505.el9.x86_64 2024-09-16T11:34:19.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T11:34:20.019 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2024-09-16T11:34:21.296 INFO:teuthology.orchestra.run.smithi070.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T11:34:21.316 INFO:teuthology.orchestra.run.smithi070.stderr:done 2024-09-16T11:34:21.318 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-16T11:34:21.333 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.nDp5cy7hLd 2024-09-16T11:34:21.334 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nDp5cy7hLd 2024-09-16T11:34:21.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.nDp5cy7hLd 2024-09-16T11:34:21.539 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.nDp5cy7hLd is 6KB 2024-09-16T11:34:21.588 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.nDp5cy7hLd 2024-09-16T11:34:21.602 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T11:34:21.665 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T11:34:21.665 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T11:34:21.665 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T11:34:21.665 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T11:34:21.667 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T11:34:21.822 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2024-09-16T11:34:31.685 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T11:34:31.725 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T11:34:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T11:34:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T11:34:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T11:34:32.050 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T11:34:32.896 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T11:34:32.896 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:34:32.896 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:34:32.896 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T11:34:32.896 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T11:34:32.897 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T11:34:32.897 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T11:34:32.897 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:34:32.897 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:34:33.249 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T11:34:33.335 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T11:34:33.335 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T11:34:33.335 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T11:34:33.335 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T11:34:33.335 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T11:34:33.336 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T11:34:33.336 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi192.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T11:34:33.336 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T11:34:33.394 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T11:34:33.394 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T11:34:33.395 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -qi grub2-tools 2024-09-16T11:34:33.426 INFO:teuthology.orchestra.run.smithi192.stdout:Name : grub2-tools 2024-09-16T11:34:33.426 INFO:teuthology.orchestra.run.smithi192.stdout:Epoch : 1 2024-09-16T11:34:33.427 INFO:teuthology.orchestra.run.smithi192.stdout:Version : 2.06 2024-09-16T11:34:33.427 INFO:teuthology.orchestra.run.smithi192.stdout:Release : 82.el9 2024-09-16T11:34:33.427 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: x86_64 2024-09-16T11:34:33.427 INFO:teuthology.orchestra.run.smithi192.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T11:34:33.427 INFO:teuthology.orchestra.run.smithi192.stdout:Group : Unspecified 2024-09-16T11:34:33.427 INFO:teuthology.orchestra.run.smithi192.stdout:Size : 8274670 2024-09-16T11:34:33.427 INFO:teuthology.orchestra.run.smithi192.stdout:License : GPLv3+ 2024-09-16T11:34:33.427 INFO:teuthology.orchestra.run.smithi192.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T11:34:33.427 INFO:teuthology.orchestra.run.smithi192.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T11:34:33.427 INFO:teuthology.orchestra.run.smithi192.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T11:34:33.427 INFO:teuthology.orchestra.run.smithi192.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T11:34:33.427 INFO:teuthology.orchestra.run.smithi192.stdout:Packager : builder@centos.org 2024-09-16T11:34:33.427 INFO:teuthology.orchestra.run.smithi192.stdout:Vendor : CentOS 2024-09-16T11:34:33.428 INFO:teuthology.orchestra.run.smithi192.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T11:34:33.428 INFO:teuthology.orchestra.run.smithi192.stdout:Summary : Support tools for GRUB. 2024-09-16T11:34:33.428 INFO:teuthology.orchestra.run.smithi192.stdout:Description : 2024-09-16T11:34:33.428 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:34:33.428 INFO:teuthology.orchestra.run.smithi192.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T11:34:33.428 INFO:teuthology.orchestra.run.smithi192.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T11:34:33.428 INFO:teuthology.orchestra.run.smithi192.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T11:34:33.428 INFO:teuthology.orchestra.run.smithi192.stdout:hardware devices. 2024-09-16T11:34:33.428 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:34:33.428 INFO:teuthology.orchestra.run.smithi192.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T11:34:33.429 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T11:34:33.429 INFO:teuthology.task.kernel:Updating grub on smithi192 to boot 5.14.0-505.el9.x86_64 2024-09-16T11:34:33.429 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T11:34:34.093 INFO:teuthology.orchestra.run.smithi192.stderr:Generating grub configuration file ... 2024-09-16T11:34:35.446 INFO:teuthology.orchestra.run.smithi192.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T11:34:35.461 INFO:teuthology.orchestra.run.smithi192.stderr:done 2024-09-16T11:34:35.473 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-09-16T11:34:35.488 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.KjKcQrv9Im 2024-09-16T11:34:35.488 DEBUG:teuthology.orchestra.run.smithi192:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.KjKcQrv9Im 2024-09-16T11:34:35.556 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.KjKcQrv9Im 2024-09-16T11:34:35.695 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.KjKcQrv9Im is 6KB 2024-09-16T11:34:35.743 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.KjKcQrv9Im 2024-09-16T11:34:35.758 DEBUG:teuthology.orchestra.run.smithi192:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T11:34:35.821 INFO:teuthology.orchestra.run.smithi192.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T11:34:35.821 INFO:teuthology.orchestra.run.smithi192.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T11:34:35.821 INFO:teuthology.orchestra.run.smithi192.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T11:34:35.821 INFO:teuthology.orchestra.run.smithi192.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T11:34:35.822 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T11:34:35.981 DEBUG:teuthology.orchestra.run.smithi192:> sudo shutdown -r now 2024-09-16T11:34:51.838 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T11:34:51.839 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-16T11:34:51.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:35:05.997 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T11:35:05.997 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-16T11:35:05.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:35:40.605 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-09-16T11:35:49.613 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-16T11:35:49.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:35:51.844 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T11:36:00.853 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-16T11:36:00.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:36:01.092 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-16T11:36:01.495 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-16T11:36:01.495 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T11:36:01.495 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-16T11:36:01.512 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-505.el9.x86_64 2024-09-16T11:36:01.512 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T11:36:01.513 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T11:36:01.516 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T11:36:02.517 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T11:36:02.517 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T11:36:02.619 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2024-09-16T11:36:02.639 DEBUG:teuthology.parallel:result is None 2024-09-16T11:36:07.997 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-09-16T11:36:19.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-16T11:36:19.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:36:20.322 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-09-16T11:36:20.705 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-16T11:36:20.705 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-16T11:36:20.705 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-09-16T11:36:20.721 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-505.el9.x86_64 2024-09-16T11:36:20.721 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T11:36:20.721 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T11:36:20.722 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T11:36:21.723 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T11:36:21.723 DEBUG:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T11:36:21.827 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2024-09-16T11:36:21.851 DEBUG:teuthology.parallel:result is None 2024-09-16T11:36:21.851 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T11:36:21.862 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T11:36:21.862 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T11:36:21.866 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T11:36:21.883 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T11:36:21.909 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T11:36:21.950 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T11:36:21.951 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T11:36:21.953 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T11:36:21.999 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T11:36:22.007 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T11:36:22.008 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T11:36:22.036 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T11:36:22.066 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T11:36:22.070 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T11:36:22.078 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T11:36:22.081 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T11:36:22.083 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T11:36:22.090 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T11:36:22.090 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T11:36:22.122 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T11:36:22.152 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T11:36:22.160 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T11:36:22.160 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T11:36:22.190 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T11:36:22.209 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T11:36:22.289 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T11:36:22.341 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T11:36:22.424 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T11:36:22.465 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:36:22.465 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T11:36:22.529 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T11:36:22.572 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T11:36:22.618 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T11:36:22.706 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T11:36:22.747 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:36:22.747 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T11:36:22.815 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2024-09-16T11:36:22.818 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-09-16T11:36:22.873 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T11:36:22.905 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T11:36:23.374 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T11:36:23.382 INFO:teuthology.task.internal:Starting timer... 2024-09-16T11:36:23.382 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T11:36:23.429 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T11:36:23.470 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2024-09-16T11:36:23.524 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2024-09-16T11:36:23.526 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-09-16T11:36:23.575 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-09-16T11:36:23.577 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T11:36:23.577 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2024-09-16T11:36:23.636 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2024-09-16T11:36:23.637 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-09-16T11:36:23.662 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-09-16T11:36:23.663 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi070.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-09-16T11:36:23.663 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T11:36:23.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:36:23.690 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T11:36:23.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:36:23.727 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T11:36:23.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2024-09-16T11:36:23.765 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-09-16T11:36:23.835 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T11:36:23.845 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-16T11:36:23.848 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-16T11:36:23.962 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T11:36:23.973 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T11:36:23.974 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi070.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T11:40:06.031 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi070.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-09-16T11:40:06.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-16T11:40:06.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:40:06.111 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-16T11:40:06.201 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-16T11:40:06.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-16T11:40:06.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:40:06.283 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-09-16T11:40:06.364 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-16T11:40:06.365 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T11:40:06.378 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T11:40:06.378 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T11:40:06.379 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T11:40:06.381 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T11:40:06.381 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T11:40:06.407 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T11:40:06.420 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T11:40:06.435 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T11:40:06.444 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2024-09-16T11:40:06.450 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T11:40:06.455 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2024-09-16T11:40:06.469 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T11:40:06.475 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-09-16T11:40:06.483 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T11:40:06.486 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-09-16T11:40:06.501 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T11:40:06.515 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T11:40:06.541 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-16T11:40:06.545 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T11:40:06.545 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-16T11:40:06.545 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T11:40:06.545 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T11:40:06.545 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T11:40:06.545 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T11:40:06.576 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-09-16T11:40:06.579 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T11:40:06.579 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-09-16T11:40:06.579 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T11:40:06.579 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T11:40:06.579 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T11:40:06.579 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T11:40:06.580 INFO:teuthology.run_tasks:Running task install... 2024-09-16T11:40:06.588 DEBUG:teuthology.task.install:project ceph 2024-09-16T11:40:06.588 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-16T11:40:06.588 DEBUG:teuthology.task.install:config {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'} 2024-09-16T11:40:06.588 INFO:teuthology.task.install:Using flavor: default 2024-09-16T11:40:06.594 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T11:40:06.594 INFO:teuthology.task.install:extra packages: [] 2024-09-16T11:40:06.595 DEBUG:teuthology.orchestra.run.smithi070:> sudo dnf -y install 'dnf-command(copr)' 2024-09-16T11:40:06.595 DEBUG:teuthology.orchestra.run.smithi192:> sudo dnf -y install 'dnf-command(copr)' 2024-09-16T11:40:07.147 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:46 ago on Mon 16 Sep 2024 11:39:20 AM UTC. 2024-09-16T11:40:07.171 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:46 ago on Mon 16 Sep 2024 11:39:20 AM UTC. 2024-09-16T11:40:07.233 INFO:teuthology.orchestra.run.smithi070.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-09-16T11:40:07.258 INFO:teuthology.orchestra.run.smithi192.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-09-16T11:40:07.275 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:40:07.276 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T11:40:07.276 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:40:07.299 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:40:07.301 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-16T11:40:07.301 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:40:07.336 DEBUG:teuthology.orchestra.run.smithi070:> sudo dnf -y copr enable ceph/el9 2024-09-16T11:40:07.368 DEBUG:teuthology.orchestra.run.smithi192:> sudo dnf -y copr enable ceph/el9 2024-09-16T11:40:07.508 INFO:teuthology.orchestra.run.smithi070.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-09-16T11:40:07.508 INFO:teuthology.orchestra.run.smithi070.stderr:of the main distribution, and quality may vary. 2024-09-16T11:40:07.508 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-09-16T11:40:07.508 INFO:teuthology.orchestra.run.smithi070.stderr:The Fedora Project does not exercise any power over the contents of 2024-09-16T11:40:07.508 INFO:teuthology.orchestra.run.smithi070.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-09-16T11:40:07.508 INFO:teuthology.orchestra.run.smithi070.stderr:, 2024-09-16T11:40:07.509 INFO:teuthology.orchestra.run.smithi070.stderr:and packages are not held to any quality or security level. 2024-09-16T11:40:07.509 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-09-16T11:40:07.509 INFO:teuthology.orchestra.run.smithi070.stderr:Please do not file bug reports about these packages in Fedora 2024-09-16T11:40:07.509 INFO:teuthology.orchestra.run.smithi070.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-09-16T11:40:07.538 INFO:teuthology.orchestra.run.smithi192.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-09-16T11:40:07.538 INFO:teuthology.orchestra.run.smithi192.stderr:of the main distribution, and quality may vary. 2024-09-16T11:40:07.538 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-09-16T11:40:07.538 INFO:teuthology.orchestra.run.smithi192.stderr:The Fedora Project does not exercise any power over the contents of 2024-09-16T11:40:07.538 INFO:teuthology.orchestra.run.smithi192.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-09-16T11:40:07.538 INFO:teuthology.orchestra.run.smithi192.stderr:, 2024-09-16T11:40:07.538 INFO:teuthology.orchestra.run.smithi192.stderr:and packages are not held to any quality or security level. 2024-09-16T11:40:07.538 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-09-16T11:40:07.538 INFO:teuthology.orchestra.run.smithi192.stderr:Please do not file bug reports about these packages in Fedora 2024-09-16T11:40:07.538 INFO:teuthology.orchestra.run.smithi192.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-09-16T11:40:07.641 INFO:teuthology.orchestra.run.smithi192.stdout:Repository successfully enabled. 2024-09-16T11:40:07.644 INFO:teuthology.orchestra.run.smithi070.stdout:Repository successfully enabled. 2024-09-16T11:40:07.676 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'tag': None, 'wait_for_package': False} 2024-09-16T11:40:07.677 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T11:40:07.682 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'tag': None, 'wait_for_package': False} 2024-09-16T11:40:07.682 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T11:40:07.829 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/ 2024-09-16T11:40:07.829 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1643.gd75f2382 2024-09-16T11:40:07.836 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/ 2024-09-16T11:40:07.837 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1643.gd75f2382 2024-09-16T11:40:07.942 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T11:40:07.942 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:40:07.942 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T11:40:07.950 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T11:40:07.950 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:40:07.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T11:40:07.968 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T11:40:07.968 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d75f23820806da622f561ee5ee7551015285993f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T11:40:07.975 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T11:40:07.975 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d75f23820806da622f561ee5ee7551015285993f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T11:40:08.034 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T11:40:08.041 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T11:40:08.109 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T11:40:08.116 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T11:40:08.173 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-09-16T11:40:08.175 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-09-16T11:40:08.180 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2024-09-16T11:40:08.182 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2024-09-16T11:40:08.408 INFO:teuthology.orchestra.run.smithi070.stdout:45 files removed 2024-09-16T11:40:08.408 INFO:teuthology.orchestra.run.smithi192.stdout:45 files removed 2024-09-16T11:40:08.434 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw 2024-09-16T11:40:08.437 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw 2024-09-16T11:40:09.000 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 275 kB/s | 77 kB 00:00 2024-09-16T11:40:09.003 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 278 kB/s | 77 kB 00:00 2024-09-16T11:40:09.284 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-16T11:40:09.287 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-16T11:40:09.560 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-16T11:40:09.563 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-16T11:40:09.736 INFO:teuthology.orchestra.run.smithi070.stdout:Copr repo for el9 owned by ceph 54 kB/s | 7.6 kB 00:00 2024-09-16T11:40:09.739 INFO:teuthology.orchestra.run.smithi192.stdout:Copr repo for el9 owned by ceph 54 kB/s | 7.6 kB 00:00 2024-09-16T11:40:10.144 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 22 MB/s | 8.2 MB 00:00 2024-09-16T11:40:12.755 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.2 MB 00:02 2024-09-16T11:40:14.723 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-16T11:40:19.182 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.5 MB 00:00 2024-09-16T11:40:20.658 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-16T11:40:21.487 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-16T11:40:24.976 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 1.5 MB/s | 20 MB 00:13 2024-09-16T11:40:26.868 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T11:40:29.268 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-16T11:40:29.309 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:40:29.318 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:40:29.319 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-16T11:40:29.319 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:40:29.319 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:40:29.319 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 14 M 2024-09-16T11:40:29.319 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-09-16T11:40:29.319 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.1 M 2024-09-16T11:40:29.319 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-16T11:40:29.319 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-16T11:40:29.319 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 5.0 M 2024-09-16T11:40:29.319 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 22 M 2024-09-16T11:40:29.319 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-16T11:40:29.320 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T11:40:29.320 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 637 k 2024-09-16T11:40:29.320 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T11:40:29.320 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T11:40:29.320 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T11:40:29.320 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 458 k 2024-09-16T11:40:29.320 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T11:40:29.320 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 4.9 M 2024-09-16T11:40:29.320 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T11:40:29.320 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T11:40:29.320 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 45 k 2024-09-16T11:40:29.320 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 120 k 2024-09-16T11:40:29.321 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 156 k 2024-09-16T11:40:29.321 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T11:40:29.321 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 320 k 2024-09-16T11:40:29.321 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 295 k 2024-09-16T11:40:29.321 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 100 k 2024-09-16T11:40:29.321 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T11:40:29.321 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:40:29.321 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:40:29.321 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:40:29.322 INFO:teuthology.orchestra.run.smithi192.stdout:Install 22 Packages 2024-09-16T11:40:29.322 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 2 Packages 2024-09-16T11:40:29.322 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:40:29.322 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 58 M 2024-09-16T11:40:29.322 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:40:30.081 INFO:teuthology.orchestra.run.smithi192.stdout:(1/24): ceph-base-17.2.7-1643.gd75f2382.el9.x86 7.8 MB/s | 5.0 MB 00:00 2024-09-16T11:40:30.136 INFO:teuthology.orchestra.run.smithi192.stdout:(2/24): ceph-selinux-17.2.7-1643.gd75f2382.el9. 432 kB/s | 25 kB 00:00 2024-09-16T11:40:30.220 INFO:teuthology.orchestra.run.smithi192.stdout:(3/24): libcephfs2-17.2.7-1643.gd75f2382.el9.x8 7.5 MB/s | 637 kB 00:00 2024-09-16T11:40:30.303 INFO:teuthology.orchestra.run.smithi192.stdout:(4/24): libradosstriper1-17.2.7-1643.gd75f2382. 5.4 MB/s | 458 kB 00:00 2024-09-16T11:40:30.453 INFO:teuthology.orchestra.run.smithi192.stdout:(5/24): ceph-radosgw-17.2.7-1643.gd75f2382.el9. 14 MB/s | 14 MB 00:01 2024-09-16T11:40:30.495 INFO:teuthology.orchestra.run.smithi192.stdout:(6/24): python3-ceph-argparse-17.2.7-1643.gd75f 1.1 MB/s | 45 kB 00:00 2024-09-16T11:40:30.570 INFO:teuthology.orchestra.run.smithi192.stdout:(7/24): python3-ceph-common-17.2.7-1643.gd75f23 1.6 MB/s | 120 kB 00:00 2024-09-16T11:40:30.654 INFO:teuthology.orchestra.run.smithi192.stdout:(8/24): librgw2-17.2.7-1643.gd75f2382.el9.x86_6 14 MB/s | 4.9 MB 00:00 2024-09-16T11:40:30.679 INFO:teuthology.orchestra.run.smithi192.stdout:(9/24): python3-cephfs-17.2.7-1643.gd75f2382.el 1.4 MB/s | 156 kB 00:00 2024-09-16T11:40:30.888 INFO:teuthology.orchestra.run.smithi192.stdout:(10/24): ceph-common-17.2.7-1643.gd75f2382.el9. 16 MB/s | 22 MB 00:01 2024-09-16T11:40:30.917 INFO:teuthology.orchestra.run.smithi192.stdout:(11/24): python3-rados-17.2.7-1643.gd75f2382.el 1.2 MB/s | 320 kB 00:00 2024-09-16T11:40:30.938 INFO:teuthology.orchestra.run.smithi192.stdout:(12/24): python3-rgw-17.2.7-1643.gd75f2382.el9. 1.9 MB/s | 100 kB 00:00 2024-09-16T11:40:30.951 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-16T11:40:30.964 INFO:teuthology.orchestra.run.smithi192.stdout:(13/24): python3-rbd-17.2.7-1643.gd75f2382.el9. 1.0 MB/s | 295 kB 00:00 2024-09-16T11:40:31.039 INFO:teuthology.orchestra.run.smithi192.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-09-16T11:40:31.064 INFO:teuthology.orchestra.run.smithi192.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-16T11:40:31.128 INFO:teuthology.orchestra.run.smithi192.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.2 MB/s | 292 kB 00:00 2024-09-16T11:40:31.157 INFO:teuthology.orchestra.run.smithi192.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 459 kB/s | 42 kB 00:00 2024-09-16T11:40:31.207 INFO:teuthology.orchestra.run.smithi192.stdout:(18/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.8 MB/s | 308 kB 00:00 2024-09-16T11:40:31.232 INFO:teuthology.orchestra.run.smithi192.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 647 kB/s | 49 kB 00:00 2024-09-16T11:40:31.257 INFO:teuthology.orchestra.run.smithi192.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-16T11:40:31.283 INFO:teuthology.orchestra.run.smithi192.stdout:(21/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.0 MB/s | 662 kB 00:00 2024-09-16T11:40:31.333 INFO:teuthology.orchestra.run.smithi192.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-16T11:40:31.466 INFO:teuthology.orchestra.run.smithi192.stdout:(23/24): librbd1-17.2.7-1643.gd75f2382.el9.x86_ 16 MB/s | 2.9 MB 00:00 2024-09-16T11:40:31.525 INFO:teuthology.orchestra.run.smithi192.stdout:(24/24): librados2-17.2.7-1643.gd75f2382.el9.x8 12 MB/s | 3.1 MB 00:00 2024-09-16T11:40:31.532 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:40:31.533 INFO:teuthology.orchestra.run.smithi192.stdout:Total 26 MB/s | 58 MB 00:02 2024-09-16T11:40:31.589 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:40:31.622 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:40:31.622 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:40:32.105 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-16T11:40:32.293 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:40:32.294 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:40:33.518 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:40:33.876 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-16T11:40:33.985 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-16T11:40:34.078 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-16T11:40:34.156 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-16T11:40:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-16T11:40:34.274 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-16T11:40:34.320 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-16T11:40:34.350 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-16T11:40:34.789 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-16T11:40:34.829 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-16T11:40:34.887 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-16T11:40:34.923 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-16T11:40:34.966 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 10/26 2024-09-16T11:40:35.003 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-16T11:40:35.065 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-16T11:40:35.092 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-16T11:40:35.141 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-16T11:40:35.180 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-16T11:40:35.217 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-16T11:40:35.250 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-16T11:40:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-16T11:40:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-16T11:40:35.677 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-16T11:40:35.751 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 19/26 2024-09-16T11:40:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 20/26 2024-09-16T11:40:37.578 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T11:40:38.215 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-16T11:40:38.253 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-16T11:40:38.800 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-16T11:40:38.839 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-16T11:40:38.937 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-16T11:40:38.937 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T11:40:38.937 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T11:40:38.937 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:40:38.965 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-16T11:40:40.070 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:40:40.086 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:40:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T11:40:40.086 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:40:40.086 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:40:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 14 M 2024-09-16T11:40:40.086 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2024-09-16T11:40:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.1 M 2024-09-16T11:40:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-16T11:40:40.086 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T11:40:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 5.0 M 2024-09-16T11:40:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 22 M 2024-09-16T11:40:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-16T11:40:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T11:40:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 637 k 2024-09-16T11:40:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T11:40:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T11:40:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T11:40:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 458 k 2024-09-16T11:40:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T11:40:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 4.9 M 2024-09-16T11:40:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T11:40:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T11:40:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 45 k 2024-09-16T11:40:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 120 k 2024-09-16T11:40:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 156 k 2024-09-16T11:40:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T11:40:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 320 k 2024-09-16T11:40:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 295 k 2024-09-16T11:40:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 100 k 2024-09-16T11:40:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T11:40:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:40:40.088 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:40:40.088 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:40:40.088 INFO:teuthology.orchestra.run.smithi070.stdout:Install 22 Packages 2024-09-16T11:40:40.088 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 2 Packages 2024-09-16T11:40:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:40:40.089 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 58 M 2024-09-16T11:40:40.089 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:40:40.787 INFO:teuthology.orchestra.run.smithi070.stdout:(1/24): ceph-base-17.2.7-1643.gd75f2382.el9.x86 10 MB/s | 5.0 MB 00:00 2024-09-16T11:40:40.837 INFO:teuthology.orchestra.run.smithi070.stdout:(2/24): ceph-selinux-17.2.7-1643.gd75f2382.el9. 505 kB/s | 25 kB 00:00 2024-09-16T11:40:40.937 INFO:teuthology.orchestra.run.smithi070.stdout:(3/24): libcephfs2-17.2.7-1643.gd75f2382.el9.x8 6.2 MB/s | 637 kB 00:00 2024-09-16T11:40:41.029 INFO:teuthology.orchestra.run.smithi070.stdout:(4/24): libradosstriper1-17.2.7-1643.gd75f2382. 4.9 MB/s | 458 kB 00:00 2024-09-16T11:40:41.530 INFO:teuthology.orchestra.run.smithi070.stdout:(5/24): ceph-common-17.2.7-1643.gd75f2382.el9.x 19 MB/s | 22 MB 00:01 2024-09-16T11:40:41.596 INFO:teuthology.orchestra.run.smithi070.stdout:(6/24): librgw2-17.2.7-1643.gd75f2382.el9.x86_6 8.7 MB/s | 4.9 MB 00:00 2024-09-16T11:40:41.621 INFO:teuthology.orchestra.run.smithi070.stdout:(7/24): python3-ceph-argparse-17.2.7-1643.gd75f 493 kB/s | 45 kB 00:00 2024-09-16T11:40:41.647 INFO:teuthology.orchestra.run.smithi070.stdout:(8/24): python3-ceph-common-17.2.7-1643.gd75f23 2.3 MB/s | 120 kB 00:00 2024-09-16T11:40:41.797 INFO:teuthology.orchestra.run.smithi070.stdout:(9/24): ceph-radosgw-17.2.7-1643.gd75f2382.el9. 9.7 MB/s | 14 MB 00:01 2024-09-16T11:40:41.822 INFO:teuthology.orchestra.run.smithi070.stdout:(10/24): python3-cephfs-17.2.7-1643.gd75f2382.e 779 kB/s | 156 kB 00:00 2024-09-16T11:40:41.856 INFO:teuthology.orchestra.run.smithi070.stdout:(11/24): python3-rados-17.2.7-1643.gd75f2382.el 1.5 MB/s | 320 kB 00:00 2024-09-16T11:40:41.881 INFO:teuthology.orchestra.run.smithi070.stdout:(12/24): python3-rbd-17.2.7-1643.gd75f2382.el9. 3.5 MB/s | 295 kB 00:00 2024-09-16T11:40:41.906 INFO:teuthology.orchestra.run.smithi070.stdout:(13/24): python3-rgw-17.2.7-1643.gd75f2382.el9. 1.2 MB/s | 100 kB 00:00 2024-09-16T11:40:42.465 INFO:teuthology.orchestra.run.smithi070.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 77 kB/s | 45 kB 00:00 2024-09-16T11:40:42.657 INFO:teuthology.orchestra.run.smithi070.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 200 kB/s | 160 kB 00:00 2024-09-16T11:40:42.774 INFO:teuthology.orchestra.run.smithi070.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 764 kB/s | 662 kB 00:00 2024-09-16T11:40:42.873 INFO:teuthology.orchestra.run.smithi070.stdout:(17/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.1 MB/s | 308 kB 00:00 2024-09-16T11:40:42.899 INFO:teuthology.orchestra.run.smithi070.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 2.6 MB/s | 49 kB 00:00 2024-09-16T11:40:42.933 INFO:teuthology.orchestra.run.smithi070.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-16T11:40:43.008 INFO:teuthology.orchestra.run.smithi070.stdout:(20/24): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-16T11:40:43.258 INFO:teuthology.orchestra.run.smithi070.stdout:(21/24): librados2-17.2.7-1643.gd75f2382.el9.x8 12 MB/s | 3.1 MB 00:00 2024-09-16T11:40:43.275 INFO:teuthology.orchestra.run.smithi070.stdout:(22/24): python3-prettytable-0.7.2-27.el9.noarc 68 kB/s | 42 kB 00:00 2024-09-16T11:40:43.300 INFO:teuthology.orchestra.run.smithi070.stdout:(23/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 350 kB/s | 292 kB 00:00 2024-09-16T11:40:43.442 INFO:teuthology.orchestra.run.smithi070.stdout:(24/24): librbd1-17.2.7-1643.gd75f2382.el9.x86_ 16 MB/s | 2.9 MB 00:00 2024-09-16T11:40:43.446 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:40:43.447 INFO:teuthology.orchestra.run.smithi070.stdout:Total 17 MB/s | 58 MB 00:03 2024-09-16T11:40:43.508 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:40:43.542 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:40:43.543 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:40:44.210 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:40:44.211 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:40:45.515 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:40:45.890 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-16T11:40:45.955 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-16T11:40:46.030 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-16T11:40:46.116 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-16T11:40:46.139 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-16T11:40:46.212 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-16T11:40:46.257 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-16T11:40:46.286 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-16T11:40:46.726 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-16T11:40:46.751 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-16T11:40:46.817 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-16T11:40:46.850 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-16T11:40:46.893 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 10/26 2024-09-16T11:40:46.930 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-16T11:40:46.992 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-16T11:40:47.020 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-16T11:40:47.069 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-16T11:40:47.125 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-16T11:40:47.162 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-16T11:40:47.195 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-16T11:40:47.489 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-16T11:40:47.524 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-16T11:40:47.605 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-16T11:40:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 19/26 2024-09-16T11:40:47.710 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 20/26 2024-09-16T11:40:48.971 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-16T11:40:48.972 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-09-16T11:40:48.972 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-09-16T11:40:48.972 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-09-16T11:40:48.972 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-09-16T11:40:48.972 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-09-16T11:40:48.972 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-09-16T11:40:48.972 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-09-16T11:40:48.972 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:40:50.113 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-16T11:40:50.240 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-16T11:40:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-16T11:40:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-16T11:40:50.401 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:40:50.401 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T11:40:50.401 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T11:40:50.401 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T11:40:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:40:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-16T11:40:50.491 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-16T11:40:50.491 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T11:40:50.679 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-16T11:40:50.719 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-16T11:40:50.807 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-16T11:40:50.807 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T11:40:50.808 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T11:40:50.808 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:40:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-16T11:40:51.764 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T11:40:51.764 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 1/26 2024-09-16T11:40:51.764 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-16T11:40:51.764 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-16T11:40:51.765 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-16T11:40:51.765 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 5/26 2024-09-16T11:40:51.765 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 6/26 2024-09-16T11:40:51.765 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 7/26 2024-09-16T11:40:51.765 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 8/26 2024-09-16T11:40:51.765 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 9/26 2024-09-16T11:40:51.765 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 10/26 2024-09-16T11:40:51.765 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-16T11:40:51.765 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-16T11:40:51.765 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 13/26 2024-09-16T11:40:51.765 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-16T11:40:51.765 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-16T11:40:51.765 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-16T11:40:51.766 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-16T11:40:51.766 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-16T11:40:51.766 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-16T11:40:51.766 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-16T11:40:51.766 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-16T11:40:51.766 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-16T11:40:51.766 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-16T11:40:51.766 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-16T11:40:51.766 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 25/26 2024-09-16T11:40:52.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T11:40:52.656 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:40:52.656 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-09-16T11:40:52.656 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.656 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.656 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:40:52.656 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.656 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.656 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.656 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.656 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T11:40:52.656 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.656 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T11:40:52.657 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T11:40:52.657 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T11:40:52.657 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.657 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T11:40:52.657 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.657 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T11:40:52.657 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T11:40:52.657 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.657 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.657 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.657 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T11:40:52.657 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.657 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.658 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:40:52.658 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T11:40:52.658 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:40:52.658 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:40:53.019 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-test 2024-09-16T11:40:53.585 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:27 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:40:53.716 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:40:53.717 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:40:53.717 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-16T11:40:53.717 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:40:53.717 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:40:53.717 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 48 M 2024-09-16T11:40:53.717 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-16T11:40:53.717 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 163 k 2024-09-16T11:40:53.717 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T11:40:53.718 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T11:40:53.718 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:40:53.718 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:40:53.718 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:40:53.718 INFO:teuthology.orchestra.run.smithi192.stdout:Install 4 Packages 2024-09-16T11:40:53.718 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:40:53.718 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 48 M 2024-09-16T11:40:53.719 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 193 M 2024-09-16T11:40:53.719 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:40:54.182 INFO:teuthology.orchestra.run.smithi192.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.6 MB/s | 304 kB 00:00 2024-09-16T11:40:54.207 INFO:teuthology.orchestra.run.smithi192.stdout:(2/4): libcephsqlite-17.2.7-1643.gd75f2382.el9. 774 kB/s | 163 kB 00:00 2024-09-16T11:40:54.224 INFO:teuthology.orchestra.run.smithi192.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-16T11:40:55.483 INFO:teuthology.orchestra.run.smithi192.stdout:(4/4): ceph-test-17.2.7-1643.gd75f2382.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-16T11:40:55.485 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:40:55.485 INFO:teuthology.orchestra.run.smithi192.stdout:Total 27 MB/s | 48 MB 00:01 2024-09-16T11:40:55.512 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:40:55.531 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:40:55.531 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:40:55.896 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:40:55.897 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:40:56.435 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:40:56.525 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T11:40:56.565 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T11:40:56.701 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-16T11:41:00.689 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-16T11:41:00.690 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2024-09-16T11:41:00.690 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2024-09-16T11:41:00.690 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2024-09-16T11:41:00.690 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2024-09-16T11:41:00.690 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2024-09-16T11:41:00.690 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /root 2024-09-16T11:41:00.690 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2024-09-16T11:41:00.690 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:01.133 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-16T11:41:01.215 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-16T11:41:02.015 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-16T11:41:02.093 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-16T11:41:02.093 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:41:02.093 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T11:41:02.093 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T11:41:02.094 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T11:41:02.094 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:02.095 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-16T11:41:02.184 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-16T11:41:02.184 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T11:41:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-16T11:41:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 1/4 2024-09-16T11:41:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2/4 2024-09-16T11:41:03.319 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T11:41:03.510 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T11:41:03.511 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 1/26 2024-09-16T11:41:03.511 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-16T11:41:03.511 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-16T11:41:03.511 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-16T11:41:03.511 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 5/26 2024-09-16T11:41:03.511 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 6/26 2024-09-16T11:41:03.511 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 7/26 2024-09-16T11:41:03.511 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 8/26 2024-09-16T11:41:03.511 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 9/26 2024-09-16T11:41:03.511 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 10/26 2024-09-16T11:41:03.511 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-16T11:41:03.511 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-16T11:41:03.512 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 13/26 2024-09-16T11:41:03.512 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-16T11:41:03.512 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-16T11:41:03.512 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-16T11:41:03.512 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-16T11:41:03.512 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-16T11:41:03.512 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-16T11:41:03.512 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-16T11:41:03.512 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-16T11:41:03.512 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-16T11:41:03.512 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-16T11:41:03.512 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-16T11:41:03.512 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 25/26 2024-09-16T11:41:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T11:41:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:03.640 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:41:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:03.640 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T11:41:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T11:41:03.641 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:03.641 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:03.914 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph 2024-09-16T11:41:04.492 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T11:41:04.493 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:04.493 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2024-09-16T11:41:04.493 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.493 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.493 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:41:04.493 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.493 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.493 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.493 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.493 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T11:41:04.493 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.493 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T11:41:04.494 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T11:41:04.494 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T11:41:04.494 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.494 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T11:41:04.494 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.494 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T11:41:04.494 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T11:41:04.494 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.494 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.494 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.494 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T11:41:04.494 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.494 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.495 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:04.495 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T11:41:04.495 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:04.495 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:41:04.495 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:38 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:04.635 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:04.638 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-16T11:41:04.638 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:04.638 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:41:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 6.5 k 2024-09-16T11:41:04.638 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-16T11:41:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.1 M 2024-09-16T11:41:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 1.4 M 2024-09-16T11:41:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 240 k 2024-09-16T11:41:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.9 M 2024-09-16T11:41:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 15 M 2024-09-16T11:41:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T11:41:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T11:41:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T11:41:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T11:41:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T11:41:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T11:41:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T11:41:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections 2024-09-16T11:41:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T11:41:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T11:41:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T11:41:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T11:41:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T11:41:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T11:41:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T11:41:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T11:41:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T11:41:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T11:41:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T11:41:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T11:41:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T11:41:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T11:41:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T11:41:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T11:41:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T11:41:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T11:41:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T11:41:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T11:41:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T11:41:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:04.642 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:41:04.642 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:04.642 INFO:teuthology.orchestra.run.smithi192.stdout:Install 34 Packages 2024-09-16T11:41:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:04.642 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 27 M 2024-09-16T11:41:04.643 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 96 M 2024-09-16T11:41:04.643 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:41:04.864 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-test 2024-09-16T11:41:05.274 INFO:teuthology.orchestra.run.smithi192.stdout:(1/34): ceph-17.2.7-1643.gd75f2382.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-16T11:41:05.428 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:28 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:41:05.482 INFO:teuthology.orchestra.run.smithi192.stdout:(2/34): ceph-mds-17.2.7-1643.gd75f2382.el9.x86_ 6.1 MB/s | 2.1 MB 00:00 2024-09-16T11:41:05.516 INFO:teuthology.orchestra.run.smithi192.stdout:(3/34): ceph-mgr-17.2.7-1643.gd75f2382.el9.x86_ 3.8 MB/s | 1.4 MB 00:00 2024-09-16T11:41:05.565 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:41:05.566 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T11:41:05.566 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:05.566 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:41:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 48 M 2024-09-16T11:41:05.566 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T11:41:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 163 k 2024-09-16T11:41:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T11:41:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T11:41:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:05.567 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:41:05.567 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:05.567 INFO:teuthology.orchestra.run.smithi070.stdout:Install 4 Packages 2024-09-16T11:41:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:05.568 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 48 M 2024-09-16T11:41:05.569 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 193 M 2024-09-16T11:41:05.569 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:41:05.608 INFO:teuthology.orchestra.run.smithi192.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1643.gd75f 2.6 MB/s | 240 kB 00:00 2024-09-16T11:41:05.675 INFO:teuthology.orchestra.run.smithi192.stdout:(5/34): ceph-mon-17.2.7-1643.gd75f2382.el9.x86_ 9.8 MB/s | 3.9 MB 00:00 2024-09-16T11:41:05.792 INFO:teuthology.orchestra.run.smithi192.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-16T11:41:05.834 INFO:teuthology.orchestra.run.smithi192.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-16T11:41:05.873 INFO:teuthology.orchestra.run.smithi070.stdout:(1/4): libcephsqlite-17.2.7-1643.gd75f2382.el9. 633 kB/s | 163 kB 00:00 2024-09-16T11:41:05.984 INFO:teuthology.orchestra.run.smithi192.stdout:(8/34): ceph-osd-17.2.7-1643.gd75f2382.el9.x86_ 31 MB/s | 15 MB 00:00 2024-09-16T11:41:06.009 INFO:teuthology.orchestra.run.smithi192.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 772 kB/s | 135 kB 00:00 2024-09-16T11:41:06.051 INFO:teuthology.orchestra.run.smithi192.stdout:(10/34): python3-cryptography-36.0.1-4.el9.x86_ 3.3 MB/s | 1.2 MB 00:00 2024-09-16T11:41:06.076 INFO:teuthology.orchestra.run.smithi192.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-09-16T11:41:06.115 INFO:teuthology.orchestra.run.smithi070.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 609 kB/s | 304 kB 00:00 2024-09-16T11:41:06.118 INFO:teuthology.orchestra.run.smithi192.stdout:(12/34): python3-requests-2.25.1-8.el9.noarch.r 933 kB/s | 125 kB 00:00 2024-09-16T11:41:06.193 INFO:teuthology.orchestra.run.smithi192.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 297 kB/s | 35 kB 00:00 2024-09-16T11:41:06.210 INFO:teuthology.orchestra.run.smithi192.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-16T11:41:06.223 INFO:teuthology.orchestra.run.smithi070.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 182 kB/s | 64 kB 00:00 2024-09-16T11:41:06.236 INFO:teuthology.orchestra.run.smithi192.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 748 kB/s | 77 kB 00:00 2024-09-16T11:41:06.261 INFO:teuthology.orchestra.run.smithi192.stdout:(16/34): python3-pytz-2021.1-5.el9.noarch.rpm 765 kB/s | 51 kB 00:00 2024-09-16T11:41:06.286 INFO:teuthology.orchestra.run.smithi192.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-09-16T11:41:06.336 INFO:teuthology.orchestra.run.smithi192.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 580 kB/s | 43 kB 00:00 2024-09-16T11:41:06.353 INFO:teuthology.orchestra.run.smithi192.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 735 kB/s | 11 kB 00:00 2024-09-16T11:41:06.370 INFO:teuthology.orchestra.run.smithi192.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-16T11:41:06.404 INFO:teuthology.orchestra.run.smithi192.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 350 kB/s | 18 kB 00:00 2024-09-16T11:41:06.421 INFO:teuthology.orchestra.run.smithi192.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 459 kB/s | 23 kB 00:00 2024-09-16T11:41:06.438 INFO:teuthology.orchestra.run.smithi192.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 581 kB/s | 19 kB 00:00 2024-09-16T11:41:06.455 INFO:teuthology.orchestra.run.smithi192.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 590 kB/s | 20 kB 00:00 2024-09-16T11:41:06.480 INFO:teuthology.orchestra.run.smithi192.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-09-16T11:41:06.505 INFO:teuthology.orchestra.run.smithi192.stdout:(26/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-09-16T11:41:06.531 INFO:teuthology.orchestra.run.smithi192.stdout:(27/34): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-16T11:41:06.548 INFO:teuthology.orchestra.run.smithi192.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 391 kB/s | 16 kB 00:00 2024-09-16T11:41:06.565 INFO:teuthology.orchestra.run.smithi192.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.6 MB/s | 90 kB 00:00 2024-09-16T11:41:06.581 INFO:teuthology.orchestra.run.smithi192.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 1.0 MB/s | 36 kB 00:00 2024-09-16T11:41:06.607 INFO:teuthology.orchestra.run.smithi192.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-16T11:41:06.641 INFO:teuthology.orchestra.run.smithi192.stdout:(32/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-16T11:41:06.666 INFO:teuthology.orchestra.run.smithi192.stdout:(33/34): python3-zc-lockfile-2.0-10.el9.noarch. 340 kB/s | 20 kB 00:00 2024-09-16T11:41:06.691 INFO:teuthology.orchestra.run.smithi192.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-16T11:41:06.697 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:06.697 INFO:teuthology.orchestra.run.smithi192.stdout:Total 13 MB/s | 27 MB 00:02 2024-09-16T11:41:06.835 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:41:06.873 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:41:06.874 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:41:07.107 INFO:teuthology.orchestra.run.smithi070.stdout:(4/4): ceph-test-17.2.7-1643.gd75f2382.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-16T11:41:07.109 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:07.109 INFO:teuthology.orchestra.run.smithi070.stdout:Total 31 MB/s | 48 MB 00:01 2024-09-16T11:41:07.137 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:41:07.156 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:41:07.156 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:41:07.271 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:41:07.271 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:41:07.520 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:41:07.520 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:41:08.034 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:41:08.119 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T11:41:08.168 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T11:41:08.192 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-16T11:41:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:41:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-16T11:41:08.609 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-16T11:41:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-16T11:41:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-16T11:41:08.812 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-16T11:41:08.853 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-16T11:41:08.897 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-16T11:41:08.922 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-16T11:41:08.948 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-16T11:41:08.981 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-16T11:41:09.022 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-16T11:41:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-16T11:41:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-16T11:41:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-16T11:41:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-16T11:41:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-16T11:41:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-16T11:41:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-16T11:41:09.524 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-16T11:41:09.563 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-16T11:41:09.611 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-16T11:41:09.668 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-16T11:41:09.854 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-16T11:41:09.919 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-16T11:41:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-16T11:41:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-16T11:41:10.132 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-16T11:41:10.248 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-16T11:41:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 29/34 2024-09-16T11:41:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-16T11:41:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-16T11:41:10.461 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:41:10.461 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T11:41:10.461 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T11:41:10.462 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T11:41:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:11.779 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-16T11:41:11.806 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-16T11:41:11.806 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:41:11.806 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T11:41:11.807 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T11:41:11.807 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T11:41:11.807 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:12.188 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-16T11:41:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-16T11:41:12.212 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:41:12.212 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T11:41:12.212 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T11:41:12.212 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T11:41:12.212 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:12.414 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-16T11:41:12.443 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-16T11:41:12.443 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:41:12.443 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T11:41:12.444 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T11:41:12.444 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T11:41:12.444 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:12.577 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-16T11:41:12.648 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-16T11:41:12.711 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-16T11:41:14.205 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-16T11:41:14.205 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 1/34 2024-09-16T11:41:14.205 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2/34 2024-09-16T11:41:14.205 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 3/34 2024-09-16T11:41:14.205 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 4/34 2024-09-16T11:41:14.205 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 5/34 2024-09-16T11:41:14.205 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 6/34 2024-09-16T11:41:14.205 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-16T11:41:14.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-16T11:41:14.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-16T11:41:14.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-16T11:41:14.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-16T11:41:14.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-16T11:41:14.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-16T11:41:14.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-16T11:41:14.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-16T11:41:14.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-16T11:41:14.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-16T11:41:14.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-16T11:41:14.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-16T11:41:14.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-16T11:41:14.207 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-16T11:41:14.207 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-16T11:41:14.207 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-16T11:41:14.207 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-16T11:41:14.207 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-16T11:41:14.207 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-16T11:41:14.207 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-16T11:41:14.207 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-16T11:41:14.208 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-16T11:41:14.208 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-16T11:41:14.208 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-16T11:41:14.208 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-16T11:41:14.208 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-16T11:41:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-16T11:41:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 1/4 2024-09-16T11:41:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2/4 2024-09-16T11:41:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T11:41:15.075 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-16T11:41:15.075 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:15.075 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:41:15.075 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:15.076 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:15.076 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:15.076 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:15.076 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:15.076 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:15.076 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T11:41:15.076 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T11:41:15.076 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T11:41:15.076 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T11:41:15.076 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T11:41:15.076 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T11:41:15.077 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T11:41:15.078 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T11:41:15.078 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T11:41:15.078 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T11:41:15.078 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T11:41:15.078 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T11:41:15.078 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T11:41:15.078 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T11:41:15.078 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T11:41:15.078 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:15.078 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T11:41:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:15.225 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:41:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T11:41:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T11:41:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:15.225 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:41:15.464 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-base 2024-09-16T11:41:15.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph 2024-09-16T11:41:16.034 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:50 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:16.093 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:39 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:41:16.124 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:41:16.166 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:16.167 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-16T11:41:16.167 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:16.227 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:41:16.230 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:16.230 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T11:41:16.230 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:16.230 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:41:16.230 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 6.5 k 2024-09-16T11:41:16.230 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T11:41:16.230 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.1 M 2024-09-16T11:41:16.230 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 1.4 M 2024-09-16T11:41:16.230 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 240 k 2024-09-16T11:41:16.230 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.9 M 2024-09-16T11:41:16.230 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 15 M 2024-09-16T11:41:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T11:41:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T11:41:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T11:41:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T11:41:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T11:41:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T11:41:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T11:41:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections 2024-09-16T11:41:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T11:41:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T11:41:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T11:41:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T11:41:16.231 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T11:41:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T11:41:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T11:41:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T11:41:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T11:41:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T11:41:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T11:41:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T11:41:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T11:41:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T11:41:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T11:41:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T11:41:16.232 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T11:41:16.233 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T11:41:16.233 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T11:41:16.233 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T11:41:16.233 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T11:41:16.233 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:16.233 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:41:16.233 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:16.233 INFO:teuthology.orchestra.run.smithi070.stdout:Install 34 Packages 2024-09-16T11:41:16.233 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:16.234 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 27 M 2024-09-16T11:41:16.234 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 96 M 2024-09-16T11:41:16.234 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:41:16.241 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install cephadm 2024-09-16T11:41:16.805 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:50 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:16.849 INFO:teuthology.orchestra.run.smithi070.stdout:(1/34): ceph-17.2.7-1643.gd75f2382.el9.x86_64.r 54 kB/s | 6.5 kB 00:00 2024-09-16T11:41:16.939 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:16.940 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-16T11:41:16.940 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:16.940 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:41:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 83 k 2024-09-16T11:41:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:16.940 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:41:16.940 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:16.940 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-16T11:41:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:16.941 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 83 k 2024-09-16T11:41:16.941 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 375 k 2024-09-16T11:41:16.941 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:41:17.032 INFO:teuthology.orchestra.run.smithi070.stdout:(2/34): ceph-mgr-17.2.7-1643.gd75f2382.el9.x86_ 4.6 MB/s | 1.4 MB 00:00 2024-09-16T11:41:17.107 INFO:teuthology.orchestra.run.smithi070.stdout:(3/34): ceph-mds-17.2.7-1643.gd75f2382.el9.x86_ 5.5 MB/s | 2.1 MB 00:00 2024-09-16T11:41:17.122 INFO:teuthology.orchestra.run.smithi192.stdout:cephadm-17.2.7-1643.gd75f2382.el9.noarch.rpm 457 kB/s | 83 kB 00:00 2024-09-16T11:41:17.122 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:17.123 INFO:teuthology.orchestra.run.smithi192.stdout:Total 453 kB/s | 83 kB 00:00 2024-09-16T11:41:17.123 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:41:17.128 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:41:17.128 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:41:17.130 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:41:17.130 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:41:17.166 INFO:teuthology.orchestra.run.smithi070.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1643.gd75f 4.0 MB/s | 240 kB 00:00 2024-09-16T11:41:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:41:17.225 INFO:teuthology.orchestra.run.smithi070.stdout:(5/34): ceph-mon-17.2.7-1643.gd75f2382.el9.x86_ 10 MB/s | 3.9 MB 00:00 2024-09-16T11:41:17.342 INFO:teuthology.orchestra.run.smithi070.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-16T11:41:17.384 INFO:teuthology.orchestra.run.smithi070.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-16T11:41:17.426 INFO:teuthology.orchestra.run.smithi070.stdout:(8/34): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-09-16T11:41:17.459 INFO:teuthology.orchestra.run.smithi070.stdout:(9/34): python3-requests-2.25.1-8.el9.noarch.rp 3.7 MB/s | 125 kB 00:00 2024-09-16T11:41:17.493 INFO:teuthology.orchestra.run.smithi070.stdout:(10/34): python3-cryptography-36.0.1-4.el9.x86_ 4.6 MB/s | 1.2 MB 00:00 2024-09-16T11:41:17.518 INFO:teuthology.orchestra.run.smithi070.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-16T11:41:17.655 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:41:17.702 INFO:teuthology.orchestra.run.smithi070.stdout:(12/34): ceph-osd-17.2.7-1643.gd75f2382.el9.x86 23 MB/s | 15 MB 00:00 2024-09-16T11:41:17.718 INFO:teuthology.orchestra.run.smithi070.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 173 kB/s | 35 kB 00:00 2024-09-16T11:41:17.750 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:41:17.762 INFO:teuthology.orchestra.run.smithi070.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 644 kB/s | 172 kB 00:00 2024-09-16T11:41:17.777 INFO:teuthology.orchestra.run.smithi070.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 868 kB/s | 51 kB 00:00 2024-09-16T11:41:17.794 INFO:teuthology.orchestra.run.smithi070.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-09-16T11:41:17.828 INFO:teuthology.orchestra.run.smithi070.stdout:(17/34): python3-packaging-20.9-5.el9.noarch.rp 612 kB/s | 77 kB 00:00 2024-09-16T11:41:17.853 INFO:teuthology.orchestra.run.smithi070.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 577 kB/s | 43 kB 00:00 2024-09-16T11:41:17.879 INFO:teuthology.orchestra.run.smithi070.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 651 kB/s | 11 kB 00:00 2024-09-16T11:41:17.904 INFO:teuthology.orchestra.run.smithi070.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-16T11:41:17.921 INFO:teuthology.orchestra.run.smithi070.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 439 kB/s | 18 kB 00:00 2024-09-16T11:41:17.938 INFO:teuthology.orchestra.run.smithi070.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 692 kB/s | 23 kB 00:00 2024-09-16T11:41:17.955 INFO:teuthology.orchestra.run.smithi070.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 580 kB/s | 19 kB 00:00 2024-09-16T11:41:17.971 INFO:teuthology.orchestra.run.smithi070.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 586 kB/s | 20 kB 00:00 2024-09-16T11:41:17.989 INFO:teuthology.orchestra.run.smithi070.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 1.3 MB/s | 46 kB 00:00 2024-09-16T11:41:18.014 INFO:teuthology.orchestra.run.smithi070.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-16T11:41:18.039 INFO:teuthology.orchestra.run.smithi070.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-09-16T11:41:18.056 INFO:teuthology.orchestra.run.smithi070.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 392 kB/s | 16 kB 00:00 2024-09-16T11:41:18.081 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:41:18.081 INFO:teuthology.orchestra.run.smithi070.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-16T11:41:18.107 INFO:teuthology.orchestra.run.smithi070.stdout:(30/34): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-09-16T11:41:18.124 INFO:teuthology.orchestra.run.smithi070.stdout:(31/34): python3-tempora-5.0.0-2.el9.noarch.rpm 532 kB/s | 36 kB 00:00 2024-09-16T11:41:18.149 INFO:teuthology.orchestra.run.smithi070.stdout:(32/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-09-16T11:41:18.166 INFO:teuthology.orchestra.run.smithi070.stdout:(33/34): python3-zc-lockfile-2.0-10.el9.noarch. 479 kB/s | 20 kB 00:00 2024-09-16T11:41:18.199 INFO:teuthology.orchestra.run.smithi070.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-09-16T11:41:18.203 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:18.203 INFO:teuthology.orchestra.run.smithi070.stdout:Total 14 MB/s | 27 MB 00:01 2024-09-16T11:41:18.325 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:41:18.326 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:18.326 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:41:18.326 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:18.326 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:18.326 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:18.329 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:41:18.366 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:41:18.366 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:41:18.510 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T11:41:18.751 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:41:18.751 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:41:19.087 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:19.223 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:19.223 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:19.223 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-09-16T11:41:19.224 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:19.224 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:41:19.224 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 137 k 2024-09-16T11:41:19.224 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:19.224 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:41:19.224 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:19.224 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-16T11:41:19.224 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:19.224 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 137 k 2024-09-16T11:41:19.224 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 379 k 2024-09-16T11:41:19.224 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:41:19.436 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-immutable-object-cache-17.2.7-1643.gd75f23 649 kB/s | 137 kB 00:00 2024-09-16T11:41:19.437 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:19.437 INFO:teuthology.orchestra.run.smithi192.stdout:Total 644 kB/s | 137 kB 00:00 2024-09-16T11:41:19.437 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:41:19.444 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:41:19.445 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:41:19.501 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:41:19.501 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:41:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:41:19.737 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-16T11:41:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-16T11:41:19.763 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:41:19.763 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T11:41:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:20.045 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:41:20.102 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-16T11:41:20.135 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-16T11:41:20.167 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-16T11:41:20.260 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-16T11:41:20.337 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-16T11:41:20.378 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-16T11:41:20.413 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-16T11:41:20.443 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-16T11:41:20.478 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-16T11:41:20.511 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-16T11:41:20.552 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-16T11:41:20.586 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-16T11:41:20.621 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-16T11:41:20.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-16T11:41:20.650 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:20.650 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:41:20.650 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:20.650 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:20.650 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:20.669 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-16T11:41:20.702 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-16T11:41:20.778 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-16T11:41:20.853 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr 2024-09-16T11:41:20.942 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-16T11:41:21.021 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-16T11:41:21.084 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-16T11:41:21.122 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-16T11:41:21.179 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-16T11:41:21.243 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-16T11:41:21.431 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-16T11:41:21.444 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:21.484 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-16T11:41:21.535 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:41:21.563 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-16T11:41:21.577 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:21.578 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-16T11:41:21.578 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:21.647 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T11:41:21.651 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-16T11:41:21.691 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-16T11:41:21.806 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-16T11:41:21.954 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 29/34 2024-09-16T11:41:21.977 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-16T11:41:22.002 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-16T11:41:22.002 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:41:22.002 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T11:41:22.003 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T11:41:22.003 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T11:41:22.003 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:22.206 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:56 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:22.339 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:22.340 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-16T11:41:22.340 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:22.340 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:41:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 3.4 M 2024-09-16T11:41:22.340 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-16T11:41:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 24 k 2024-09-16T11:41:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 15 k 2024-09-16T11:41:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-16T11:41:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T11:41:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T11:41:22.341 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-09-16T11:41:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-16T11:41:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:22.341 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:41:22.341 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:22.341 INFO:teuthology.orchestra.run.smithi192.stdout:Install 7 Packages 2024-09-16T11:41:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:22.342 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 3.7 M 2024-09-16T11:41:22.342 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 80 M 2024-09-16T11:41:22.342 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:41:22.698 INFO:teuthology.orchestra.run.smithi192.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1643.gd75f 95 kB/s | 15 kB 00:00 2024-09-16T11:41:22.715 INFO:teuthology.orchestra.run.smithi192.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1643.gd75 143 kB/s | 24 kB 00:00 2024-09-16T11:41:22.757 INFO:teuthology.orchestra.run.smithi192.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 155 kB/s | 9.0 kB 00:00 2024-09-16T11:41:22.782 INFO:teuthology.orchestra.run.smithi192.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-16T11:41:22.799 INFO:teuthology.orchestra.run.smithi192.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 486 kB/s | 41 kB 00:00 2024-09-16T11:41:22.824 INFO:teuthology.orchestra.run.smithi192.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-09-16T11:41:22.925 INFO:teuthology.orchestra.run.smithi192.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1643.gd75f2382 8.8 MB/s | 3.4 MB 00:00 2024-09-16T11:41:22.928 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:22.929 INFO:teuthology.orchestra.run.smithi192.stdout:Total 6.2 MB/s | 3.7 MB 00:00 2024-09-16T11:41:22.929 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:41:22.937 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:41:22.937 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:41:23.027 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:41:23.027 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:41:23.359 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:41:23.370 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-16T11:41:23.401 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-16T11:41:23.401 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:41:23.401 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T11:41:23.401 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T11:41:23.401 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T11:41:23.401 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-16T11:41:23.460 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-16T11:41:23.507 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-16T11:41:23.538 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-16T11:41:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 5/7 2024-09-16T11:41:23.796 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-16T11:41:23.821 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-16T11:41:23.821 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:41:23.821 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T11:41:23.821 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T11:41:23.821 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T11:41:23.821 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-16T11:41:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-16T11:41:24.054 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:41:24.054 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T11:41:24.054 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T11:41:24.055 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T11:41:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-16T11:41:24.565 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 6/7 2024-09-16T11:41:24.604 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-16T11:41:25.608 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-16T11:41:25.608 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 1/7 2024-09-16T11:41:25.608 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 2/7 2024-09-16T11:41:25.608 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 3/7 2024-09-16T11:41:25.609 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-16T11:41:25.609 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-16T11:41:25.609 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-16T11:41:25.718 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-16T11:41:25.718 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 1/34 2024-09-16T11:41:25.718 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2/34 2024-09-16T11:41:25.718 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 3/34 2024-09-16T11:41:25.718 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 4/34 2024-09-16T11:41:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 5/34 2024-09-16T11:41:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 6/34 2024-09-16T11:41:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-16T11:41:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-16T11:41:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-16T11:41:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-16T11:41:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-16T11:41:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-16T11:41:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-16T11:41:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-16T11:41:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-16T11:41:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-16T11:41:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-16T11:41:25.720 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-16T11:41:25.720 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-16T11:41:25.720 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-16T11:41:25.720 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-16T11:41:25.720 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-16T11:41:25.720 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-16T11:41:25.720 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-16T11:41:25.720 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-16T11:41:25.721 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-16T11:41:25.721 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-16T11:41:25.722 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-16T11:41:25.722 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-16T11:41:25.722 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-16T11:41:25.722 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-16T11:41:25.722 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-16T11:41:25.722 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-16T11:41:25.900 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-16T11:41:25.900 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:25.901 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:41:25.901 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:25.901 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:25.901 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:25.901 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-16T11:41:25.901 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-16T11:41:25.901 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T11:41:25.901 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T11:41:25.901 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:25.901 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:26.113 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T11:41:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-16T11:41:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:26.591 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:41:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T11:41:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T11:41:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T11:41:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T11:41:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T11:41:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T11:41:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T11:41:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T11:41:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T11:41:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T11:41:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T11:41:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T11:41:26.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T11:41:26.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T11:41:26.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T11:41:26.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T11:41:26.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T11:41:26.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T11:41:26.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T11:41:26.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T11:41:26.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T11:41:26.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T11:41:26.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T11:41:26.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T11:41:26.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T11:41:26.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T11:41:26.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T11:41:26.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T11:41:26.594 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:26.594 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:41:26.676 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:00 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:26.809 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:26.810 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-09-16T11:41:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-16T11:41:26.810 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-09-16T11:41:26.810 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:41:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 7.4 M 2024-09-16T11:41:26.810 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-16T11:41:26.810 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T11:41:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T11:41:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T11:41:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T11:41:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T11:41:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T11:41:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T11:41:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T11:41:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T11:41:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T11:41:26.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T11:41:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:26.812 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:41:26.812 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-09-16T11:41:26.812 INFO:teuthology.orchestra.run.smithi192.stdout:Install 12 Packages 2024-09-16T11:41:26.812 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:26.813 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 38 M 2024-09-16T11:41:26.813 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 212 M 2024-09-16T11:41:26.813 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:41:26.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-base 2024-09-16T11:41:27.349 INFO:teuthology.orchestra.run.smithi192.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-16 16 MB/s | 7.4 MB 00:00 2024-09-16T11:41:27.458 INFO:teuthology.orchestra.run.smithi192.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 303 kB/s | 33 kB 00:00 2024-09-16T11:41:27.517 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:50 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:41:27.607 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:41:27.649 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:41:27.649 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T11:41:27.650 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:41:27.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install cephadm 2024-09-16T11:41:27.716 INFO:teuthology.orchestra.run.smithi192.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-09-16T11:41:27.733 INFO:teuthology.orchestra.run.smithi192.stdout:(4/12): libquadmath-11.5.0-2.el9.x86_64.rpm 224 kB/s | 191 kB 00:00 2024-09-16T11:41:27.767 INFO:teuthology.orchestra.run.smithi192.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 363 kB/s | 18 kB 00:00 2024-09-16T11:41:27.850 INFO:teuthology.orchestra.run.smithi192.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 330 kB/s | 38 kB 00:00 2024-09-16T11:41:27.942 INFO:teuthology.orchestra.run.smithi192.stdout:(7/12): libgfortran-11.5.0-2.el9.x86_64.rpm 755 kB/s | 801 kB 00:01 2024-09-16T11:41:27.967 INFO:teuthology.orchestra.run.smithi192.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-16T11:41:28.043 INFO:teuthology.orchestra.run.smithi192.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 18 MB/s | 4.9 MB 00:00 2024-09-16T11:41:28.068 INFO:teuthology.orchestra.run.smithi192.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.4 MB/s | 250 kB 00:00 2024-09-16T11:41:28.272 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:51 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:41:28.401 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:41:28.402 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T11:41:28.402 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:28.402 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:41:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 83 k 2024-09-16T11:41:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:28.403 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:41:28.403 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:28.403 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T11:41:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:28.403 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 83 k 2024-09-16T11:41:28.403 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 375 k 2024-09-16T11:41:28.403 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:41:28.418 INFO:teuthology.orchestra.run.smithi192.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-16T11:41:28.576 INFO:teuthology.orchestra.run.smithi192.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-09-16T11:41:28.577 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:28.578 INFO:teuthology.orchestra.run.smithi192.stdout:Total 22 MB/s | 38 MB 00:01 2024-09-16T11:41:28.590 INFO:teuthology.orchestra.run.smithi070.stdout:cephadm-17.2.7-1643.gd75f2382.el9.noarch.rpm 443 kB/s | 83 kB 00:00 2024-09-16T11:41:28.590 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:28.591 INFO:teuthology.orchestra.run.smithi070.stdout:Total 440 kB/s | 83 kB 00:00 2024-09-16T11:41:28.591 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:41:28.596 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:41:28.596 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:41:28.597 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:41:28.598 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:41:28.684 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:41:28.932 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:41:28.958 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:41:28.958 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:41:29.139 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:41:29.219 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:41:29.454 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:41:29.455 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:41:29.595 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:41:29.859 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:41:29.859 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:29.859 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:41:29.860 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:29.860 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:29.860 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:41:30.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T11:41:30.212 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:41:30.276 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T11:41:30.322 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T11:41:30.418 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T11:41:30.459 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T11:41:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T11:41:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T11:41:30.700 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:41:30.767 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T11:41:30.835 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:41:30.836 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:30.836 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-16T11:41:30.836 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:30.836 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:41:30.837 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 137 k 2024-09-16T11:41:30.837 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:30.837 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:41:30.837 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:30.837 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T11:41:30.837 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:30.837 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 137 k 2024-09-16T11:41:30.837 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 379 k 2024-09-16T11:41:30.837 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:41:31.038 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-immutable-object-cache-17.2.7-1643.gd75f23 683 kB/s | 137 kB 00:00 2024-09-16T11:41:31.039 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:31.039 INFO:teuthology.orchestra.run.smithi070.stdout:Total 678 kB/s | 137 kB 00:00 2024-09-16T11:41:31.039 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:41:31.046 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:41:31.046 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:41:31.103 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:41:31.104 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:41:31.254 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:41:31.314 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-16T11:41:31.340 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-16T11:41:31.340 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:41:31.341 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T11:41:31.341 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:31.413 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T11:41:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T11:41:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-16T11:41:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:32.286 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:41:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:32.286 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:41:32.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr 2024-09-16T11:41:32.757 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T11:41:33.086 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:56 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:41:33.181 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:41:33.223 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:41:33.224 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T11:41:33.224 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:41:33.300 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T11:41:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T11:41:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-16T11:41:33.872 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:56 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:41:34.004 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:41:34.004 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T11:41:34.005 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:34.005 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:41:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 3.4 M 2024-09-16T11:41:34.005 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T11:41:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 24 k 2024-09-16T11:41:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 15 k 2024-09-16T11:41:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-16T11:41:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T11:41:34.005 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T11:41:34.005 INFO:teuthology.orchestra.run.smithi070.stdout:Installing weak dependencies: 2024-09-16T11:41:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-16T11:41:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:34.006 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:41:34.006 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:34.006 INFO:teuthology.orchestra.run.smithi070.stdout:Install 7 Packages 2024-09-16T11:41:34.006 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:34.007 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 3.7 M 2024-09-16T11:41:34.007 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 80 M 2024-09-16T11:41:34.007 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:41:34.227 INFO:teuthology.orchestra.run.smithi070.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1643.gd75 152 kB/s | 24 kB 00:00 2024-09-16T11:41:34.252 INFO:teuthology.orchestra.run.smithi070.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1643.gd75f 78 kB/s | 15 kB 00:00 2024-09-16T11:41:34.303 INFO:teuthology.orchestra.run.smithi070.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 120 kB/s | 9.0 kB 00:00 2024-09-16T11:41:34.328 INFO:teuthology.orchestra.run.smithi070.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-16T11:41:34.353 INFO:teuthology.orchestra.run.smithi070.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 407 kB/s | 41 kB 00:00 2024-09-16T11:41:34.378 INFO:teuthology.orchestra.run.smithi070.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-16T11:41:34.495 INFO:teuthology.orchestra.run.smithi070.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1643.gd75f2382 7.8 MB/s | 3.4 MB 00:00 2024-09-16T11:41:34.499 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:34.499 INFO:teuthology.orchestra.run.smithi070.stdout:Total 7.4 MB/s | 3.7 MB 00:00 2024-09-16T11:41:34.499 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:41:34.507 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:41:34.507 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:41:34.587 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:41:34.588 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:41:34.893 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:41:34.955 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-16T11:41:34.989 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-16T11:41:35.038 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-16T11:41:35.067 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-16T11:41:35.096 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 5/7 2024-09-16T11:41:36.084 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 6/7 2024-09-16T11:41:36.116 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-16T11:41:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-16T11:41:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 1/12 2024-09-16T11:41:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T11:41:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T11:41:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T11:41:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T11:41:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T11:41:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T11:41:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T11:41:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T11:41:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T11:41:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T11:41:36.814 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T11:41:36.814 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:36.814 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:41:36.814 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:36.815 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T11:41:36.815 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T11:41:36.815 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T11:41:36.815 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T11:41:36.815 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T11:41:36.815 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T11:41:36.815 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T11:41:36.815 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T11:41:36.815 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T11:41:36.815 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T11:41:36.815 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T11:41:36.815 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:36.815 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-16T11:41:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 1/7 2024-09-16T11:41:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 2/7 2024-09-16T11:41:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 3/7 2024-09-16T11:41:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-16T11:41:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-16T11:41:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-16T11:41:37.194 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-rook 2024-09-16T11:41:37.450 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-16T11:41:37.450 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:37.451 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:41:37.451 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:37.451 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:37.451 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:37.451 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-16T11:41:37.451 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-16T11:41:37.451 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T11:41:37.451 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T11:41:37.451 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:37.451 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:41:37.684 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T11:41:37.779 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:11 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:37.924 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:37.926 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:37.926 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-16T11:41:37.926 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:37.926 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:41:37.926 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 47 k 2024-09-16T11:41:37.927 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-16T11:41:37.927 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T11:41:37.927 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T11:41:37.927 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T11:41:37.927 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T11:41:37.927 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T11:41:37.927 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T11:41:37.927 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T11:41:37.927 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T11:41:37.927 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T11:41:37.927 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T11:41:37.927 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T11:41:37.928 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T11:41:37.928 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:37.928 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:41:37.928 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:37.928 INFO:teuthology.orchestra.run.smithi192.stdout:Install 13 Packages 2024-09-16T11:41:37.928 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:37.929 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 2.2 M 2024-09-16T11:41:37.929 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 26 M 2024-09-16T11:41:37.929 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:41:38.166 INFO:teuthology.orchestra.run.smithi192.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 161 kB/s | 19 kB 00:00 2024-09-16T11:41:38.190 INFO:teuthology.orchestra.run.smithi192.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 177 kB/s | 26 kB 00:00 2024-09-16T11:41:38.216 INFO:teuthology.orchestra.run.smithi192.stdout:(3/13): ceph-mgr-rook-17.2.7-1643.gd75f2382.el9 280 kB/s | 47 kB 00:00 2024-09-16T11:41:38.250 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:01 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:41:38.299 INFO:teuthology.orchestra.run.smithi192.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-16T11:41:38.325 INFO:teuthology.orchestra.run.smithi192.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-16T11:41:38.366 INFO:teuthology.orchestra.run.smithi192.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 803 kB/s | 54 kB 00:00 2024-09-16T11:41:38.384 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:41:38.385 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-09-16T11:41:38.385 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T11:41:38.385 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-09-16T11:41:38.385 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:41:38.385 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 7.4 M 2024-09-16T11:41:38.385 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T11:41:38.385 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T11:41:38.385 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T11:41:38.385 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T11:41:38.385 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T11:41:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T11:41:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T11:41:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T11:41:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T11:41:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T11:41:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T11:41:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T11:41:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:38.386 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:41:38.386 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-09-16T11:41:38.386 INFO:teuthology.orchestra.run.smithi070.stdout:Install 12 Packages 2024-09-16T11:41:38.386 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:38.387 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 38 M 2024-09-16T11:41:38.387 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 212 M 2024-09-16T11:41:38.387 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:41:38.408 INFO:teuthology.orchestra.run.smithi192.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 385 kB/s | 32 kB 00:00 2024-09-16T11:41:38.434 INFO:teuthology.orchestra.run.smithi192.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.3 MB/s | 279 kB 00:00 2024-09-16T11:41:38.459 INFO:teuthology.orchestra.run.smithi192.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 159 kB/s | 14 kB 00:00 2024-09-16T11:41:38.501 INFO:teuthology.orchestra.run.smithi192.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.0 MB/s | 59 kB 00:00 2024-09-16T11:41:38.526 INFO:teuthology.orchestra.run.smithi192.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-16T11:41:38.551 INFO:teuthology.orchestra.run.smithi192.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.9 MB/s | 90 kB 00:00 2024-09-16T11:41:38.627 INFO:teuthology.orchestra.run.smithi192.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.3 MB/s | 1.0 MB 00:00 2024-09-16T11:41:38.630 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:38.631 INFO:teuthology.orchestra.run.smithi192.stdout:Total 3.1 MB/s | 2.2 MB 00:00 2024-09-16T11:41:38.705 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:41:38.725 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:41:38.725 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:41:38.816 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:41:38.817 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:41:38.970 INFO:teuthology.orchestra.run.smithi070.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-16 16 MB/s | 7.4 MB 00:00 2024-09-16T11:41:39.053 INFO:teuthology.orchestra.run.smithi070.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 344 kB/s | 191 kB 00:00 2024-09-16T11:41:39.078 INFO:teuthology.orchestra.run.smithi070.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-16T11:41:39.130 INFO:teuthology.orchestra.run.smithi070.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 365 kB/s | 18 kB 00:00 2024-09-16T11:41:39.187 INFO:teuthology.orchestra.run.smithi070.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 660 kB/s | 38 kB 00:00 2024-09-16T11:41:39.313 INFO:teuthology.orchestra.run.smithi070.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 983 kB/s | 801 kB 00:00 2024-09-16T11:41:39.325 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:41:39.371 INFO:teuthology.orchestra.run.smithi070.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.4 MB/s | 3.0 MB 00:00 2024-09-16T11:41:39.484 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T11:41:39.496 INFO:teuthology.orchestra.run.smithi070.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 5.1 MB 00:00 2024-09-16T11:41:39.537 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T11:41:39.555 INFO:teuthology.orchestra.run.smithi070.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 13 MB/s | 4.9 MB 00:00 2024-09-16T11:41:39.580 INFO:teuthology.orchestra.run.smithi070.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.9 MB/s | 250 kB 00:00 2024-09-16T11:41:39.592 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T11:41:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T11:41:39.630 INFO:teuthology.orchestra.run.smithi070.stdout:(11/12): python3-devel-3.9.19-8.el9.x86_64.rpm 771 kB/s | 245 kB 00:00 2024-09-16T11:41:39.670 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T11:41:39.773 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T11:41:39.872 INFO:teuthology.orchestra.run.smithi070.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 51 MB/s | 16 MB 00:00 2024-09-16T11:41:39.873 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:39.873 INFO:teuthology.orchestra.run.smithi070.stdout:Total 26 MB/s | 38 MB 00:01 2024-09-16T11:41:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T11:41:39.922 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T11:41:40.225 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:41:40.250 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:41:40.251 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:41:40.638 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T11:41:40.685 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T11:41:40.719 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T11:41:40.744 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:41:40.745 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:41:40.765 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T11:41:40.789 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-16T11:41:41.500 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:41:41.572 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T11:41:41.608 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-16T11:41:41.608 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 1/13 2024-09-16T11:41:41.608 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T11:41:41.608 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T11:41:41.608 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T11:41:41.608 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T11:41:41.608 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T11:41:41.608 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T11:41:41.608 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T11:41:41.609 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T11:41:41.609 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T11:41:41.609 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T11:41:41.609 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T11:41:41.640 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T11:41:41.744 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T11:41:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T11:41:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T11:41:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:41.948 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:41:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T11:41:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T11:41:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T11:41:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T11:41:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T11:41:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T11:41:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T11:41:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T11:41:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T11:41:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T11:41:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T11:41:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T11:41:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:41.949 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:41.963 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T11:41:41.996 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T11:41:42.072 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T11:41:42.197 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T11:41:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T11:41:42.765 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:16 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:42.859 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T11:41:42.899 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:42.900 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:42.900 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-16T11:41:42.900 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:42.900 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:41:42.900 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 124 k 2024-09-16T11:41:42.900 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-16T11:41:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T11:41:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T11:41:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T11:41:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T11:41:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T11:41:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:42.901 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:41:42.901 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:42.901 INFO:teuthology.orchestra.run.smithi192.stdout:Install 6 Packages 2024-09-16T11:41:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:42.902 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 7.0 M 2024-09-16T11:41:42.902 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 33 M 2024-09-16T11:41:42.902 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:41:43.578 INFO:teuthology.orchestra.run.smithi192.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1643.gd75f2382.e 677 kB/s | 124 kB 00:00 2024-09-16T11:41:43.603 INFO:teuthology.orchestra.run.smithi192.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-16T11:41:43.653 INFO:teuthology.orchestra.run.smithi192.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-16T11:41:43.678 INFO:teuthology.orchestra.run.smithi192.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 767 kB/s | 58 kB 00:00 2024-09-16T11:41:43.704 INFO:teuthology.orchestra.run.smithi192.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-16T11:41:44.100 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T11:41:45.128 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T11:41:45.154 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-16T11:41:45.521 INFO:teuthology.orchestra.run.smithi192.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.8 MB/s | 6.0 MB 00:02 2024-09-16T11:41:45.523 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:45.523 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.7 MB/s | 7.0 MB 00:02 2024-09-16T11:41:45.597 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:41:45.609 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:41:45.609 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:41:45.708 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:41:45.709 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:41:46.019 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:41:46.202 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T11:41:46.320 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T11:41:46.875 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T11:41:46.958 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T11:41:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T11:41:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-16T11:41:47.748 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-16T11:41:47.748 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/6 2024-09-16T11:41:47.748 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T11:41:47.748 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T11:41:47.748 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T11:41:47.748 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T11:41:47.758 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-16T11:41:47.758 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 1/12 2024-09-16T11:41:47.758 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T11:41:47.758 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T11:41:47.758 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T11:41:47.758 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T11:41:47.758 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T11:41:47.758 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T11:41:47.758 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T11:41:47.758 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T11:41:47.758 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T11:41:47.758 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T11:41:48.037 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T11:41:48.037 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:48.037 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:41:48.037 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:48.038 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T11:41:48.038 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T11:41:48.038 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T11:41:48.038 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T11:41:48.038 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T11:41:48.038 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:48.038 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:48.131 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T11:41:48.131 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:48.131 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:41:48.132 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:48.132 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T11:41:48.132 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T11:41:48.132 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T11:41:48.132 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T11:41:48.132 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T11:41:48.132 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T11:41:48.132 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T11:41:48.132 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T11:41:48.132 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T11:41:48.132 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T11:41:48.132 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T11:41:48.133 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:48.133 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:41:48.295 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-fuse 2024-09-16T11:41:48.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-rook 2024-09-16T11:41:48.866 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:22 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:49.010 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:49.011 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:49.011 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-16T11:41:49.011 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:49.011 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:41:49.011 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 813 k 2024-09-16T11:41:49.011 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-16T11:41:49.011 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T11:41:49.012 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:49.012 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:41:49.012 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:49.012 INFO:teuthology.orchestra.run.smithi192.stdout:Install 2 Packages 2024-09-16T11:41:49.012 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:49.012 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 893 k 2024-09-16T11:41:49.012 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 2.6 M 2024-09-16T11:41:49.013 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:41:49.139 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:12 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:41:49.193 INFO:teuthology.orchestra.run.smithi192.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 501 kB/s | 80 kB 00:00 2024-09-16T11:41:49.273 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:41:49.274 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:49.274 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T11:41:49.274 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:49.274 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:41:49.274 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 47 k 2024-09-16T11:41:49.274 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T11:41:49.274 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T11:41:49.274 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T11:41:49.274 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T11:41:49.274 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T11:41:49.274 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T11:41:49.275 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T11:41:49.275 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T11:41:49.275 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T11:41:49.275 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T11:41:49.275 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T11:41:49.275 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T11:41:49.275 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T11:41:49.275 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:49.275 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:41:49.275 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:49.275 INFO:teuthology.orchestra.run.smithi070.stdout:Install 13 Packages 2024-09-16T11:41:49.275 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:49.276 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 2.2 M 2024-09-16T11:41:49.276 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 26 M 2024-09-16T11:41:49.276 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:41:49.331 INFO:teuthology.orchestra.run.smithi192.stdout:(2/2): ceph-fuse-17.2.7-1643.gd75f2382.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-09-16T11:41:49.332 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:49.332 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.7 MB/s | 893 kB 00:00 2024-09-16T11:41:49.344 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:41:49.369 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:41:49.370 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:41:49.448 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:41:49.449 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:41:49.632 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:41:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T11:41:49.821 INFO:teuthology.orchestra.run.smithi070.stdout:(1/13): ceph-mgr-rook-17.2.7-1643.gd75f2382.el9 272 kB/s | 47 kB 00:00 2024-09-16T11:41:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-16T11:41:50.180 INFO:teuthology.orchestra.run.smithi070.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 36 kB/s | 19 kB 00:00 2024-09-16T11:41:50.213 INFO:teuthology.orchestra.run.smithi070.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 45 kB/s | 26 kB 00:00 2024-09-16T11:41:50.503 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-16T11:41:50.504 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/2 2024-09-16T11:41:50.647 INFO:teuthology.orchestra.run.smithi070.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 269 kB/s | 222 kB 00:00 2024-09-16T11:41:50.772 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T11:41:50.772 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:50.772 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:41:50.772 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T11:41:50.772 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:50.772 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:50.914 INFO:teuthology.orchestra.run.smithi070.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 216 kB/s | 159 kB 00:00 2024-09-16T11:41:50.955 INFO:teuthology.orchestra.run.smithi070.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 781 kB/s | 32 kB 00:00 2024-09-16T11:41:50.981 INFO:teuthology.orchestra.run.smithi070.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 582 kB/s | 14 kB 00:00 2024-09-16T11:41:51.006 INFO:teuthology.orchestra.run.smithi070.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 8.7 MB/s | 220 kB 00:00 2024-09-16T11:41:51.031 INFO:teuthology.orchestra.run.smithi070.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 341 kB/s | 279 kB 00:00 2024-09-16T11:41:51.082 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-volume 2024-09-16T11:41:51.090 INFO:teuthology.orchestra.run.smithi070.stdout:(10/13): python3-kubernetes-26.1.0-2.el9.noarch 12 MB/s | 1.0 MB 00:00 2024-09-16T11:41:51.115 INFO:teuthology.orchestra.run.smithi070.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 712 kB/s | 59 kB 00:00 2024-09-16T11:41:51.140 INFO:teuthology.orchestra.run.smithi070.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-16T11:41:51.274 INFO:teuthology.orchestra.run.smithi070.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 86 kB/s | 54 kB 00:00 2024-09-16T11:41:51.278 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:51.278 INFO:teuthology.orchestra.run.smithi070.stdout:Total 1.1 MB/s | 2.2 MB 00:02 2024-09-16T11:41:51.345 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:41:51.360 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:41:51.360 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:41:51.448 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:41:51.449 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:41:51.661 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:25 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:51.797 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:51.798 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:51.798 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-16T11:41:51.798 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:51.798 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:41:51.798 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 270 k 2024-09-16T11:41:51.798 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:51.798 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:41:51.798 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:51.798 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-16T11:41:51.799 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:51.799 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 270 k 2024-09-16T11:41:51.799 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 1.3 M 2024-09-16T11:41:51.799 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:41:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:41:52.027 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-volume-17.2.7-1643.gd75f2382.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-16T11:41:52.027 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:52.028 INFO:teuthology.orchestra.run.smithi192.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-16T11:41:52.028 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:41:52.036 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:41:52.036 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:41:52.064 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:41:52.064 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:41:52.120 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T11:41:52.183 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T11:41:52.239 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T11:41:52.272 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:41:52.276 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T11:41:52.317 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T11:41:52.337 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:41:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:41:52.353 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:41:52.353 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T11:41:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:52.420 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T11:41:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T11:41:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T11:41:53.295 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T11:41:53.342 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T11:41:53.375 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T11:41:53.377 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:41:53.378 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:53.378 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:41:53.378 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:53.378 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:53.378 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:53.420 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T11:41:53.445 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-16T11:41:53.664 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados-devel 2024-09-16T11:41:54.245 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:28 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:54.256 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-16T11:41:54.256 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 1/13 2024-09-16T11:41:54.256 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T11:41:54.256 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T11:41:54.257 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T11:41:54.258 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T11:41:54.258 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T11:41:54.259 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T11:41:54.259 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T11:41:54.259 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T11:41:54.259 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T11:41:54.259 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T11:41:54.259 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T11:41:54.381 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:54.381 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-09-16T11:41:54.381 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:54.381 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:41:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 126 k 2024-09-16T11:41:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:54.382 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:41:54.382 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:54.382 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-16T11:41:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:54.382 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 126 k 2024-09-16T11:41:54.382 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 448 k 2024-09-16T11:41:54.382 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:41:54.573 INFO:teuthology.orchestra.run.smithi192.stdout:librados-devel-17.2.7-1643.gd75f2382.el9.x86_64 663 kB/s | 126 kB 00:00 2024-09-16T11:41:54.573 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:54.574 INFO:teuthology.orchestra.run.smithi192.stdout:Total 658 kB/s | 126 kB 00:00 2024-09-16T11:41:54.574 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:41:54.579 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:41:54.579 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:41:54.642 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:41:54.643 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:41:54.648 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T11:41:54.648 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:41:54.648 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:54.648 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T11:41:54.648 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T11:41:54.648 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T11:41:54.648 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T11:41:54.648 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T11:41:54.648 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T11:41:54.649 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T11:41:54.649 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T11:41:54.649 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T11:41:54.649 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T11:41:54.649 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T11:41:54.649 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T11:41:54.649 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:41:54.803 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:41:54.906 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T11:41:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:41:55.357 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:41:55.482 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:18 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:41:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:41:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:55.595 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:41:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:55.596 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:55.615 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:41:55.616 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T11:41:55.616 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:55.616 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:41:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 124 k 2024-09-16T11:41:55.616 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T11:41:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T11:41:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T11:41:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T11:41:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T11:41:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T11:41:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:55.617 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:41:55.617 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:41:55.617 INFO:teuthology.orchestra.run.smithi070.stdout:Install 6 Packages 2024-09-16T11:41:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:55.618 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 7.0 M 2024-09-16T11:41:55.618 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 33 M 2024-09-16T11:41:55.618 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:41:55.791 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs2 2024-09-16T11:41:56.143 INFO:teuthology.orchestra.run.smithi070.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1643.gd75f2382.e 659 kB/s | 124 kB 00:00 2024-09-16T11:41:56.176 INFO:teuthology.orchestra.run.smithi070.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-09-16T11:41:56.227 INFO:teuthology.orchestra.run.smithi070.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T11:41:56.252 INFO:teuthology.orchestra.run.smithi070.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-16T11:41:56.277 INFO:teuthology.orchestra.run.smithi070.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.0 MB/s | 548 kB 00:00 2024-09-16T11:41:56.367 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:30 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:56.460 INFO:teuthology.orchestra.run.smithi192.stdout:Package libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:41:56.505 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:56.506 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-16T11:41:56.506 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:56.610 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs-devel 2024-09-16T11:41:57.174 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:57.307 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:57.308 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:57.308 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-09-16T11:41:57.308 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:57.308 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:41:57.308 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-16T11:41:57.308 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:57.308 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:41:57.309 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:41:57.309 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-16T11:41:57.309 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:57.309 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 25 k 2024-09-16T11:41:57.309 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 110 k 2024-09-16T11:41:57.309 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:41:57.451 INFO:teuthology.orchestra.run.smithi192.stdout:libcephfs-devel-17.2.7-1643.gd75f2382.el9.x86_6 175 kB/s | 25 kB 00:00 2024-09-16T11:41:57.452 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:57.452 INFO:teuthology.orchestra.run.smithi192.stdout:Total 174 kB/s | 25 kB 00:00 2024-09-16T11:41:57.452 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:41:57.455 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:41:57.455 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:41:57.457 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:41:57.457 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:41:57.477 INFO:teuthology.orchestra.run.smithi070.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.9 MB/s | 6.0 MB 00:01 2024-09-16T11:41:57.479 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:41:57.480 INFO:teuthology.orchestra.run.smithi070.stdout:Total 3.8 MB/s | 7.0 MB 00:01 2024-09-16T11:41:57.539 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:41:57.551 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:41:57.563 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:41:57.563 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:41:57.640 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:41:57.661 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:41:57.662 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:41:57.972 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:41:57.974 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:41:58.149 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T11:41:58.195 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T11:41:58.210 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:41:58.210 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:58.210 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:41:58.210 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:41:58.211 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:41:58.211 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:58.392 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados2 2024-09-16T11:41:58.712 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T11:41:58.798 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T11:41:58.861 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T11:41:58.899 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-16T11:41:58.957 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:59.048 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:41:59.091 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:59.092 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-16T11:41:59.092 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:59.164 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librbd1 2024-09-16T11:41:59.687 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-16T11:41:59.687 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/6 2024-09-16T11:41:59.687 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T11:41:59.687 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T11:41:59.688 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T11:41:59.688 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T11:41:59.728 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:33 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:41:59.817 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:41:59.860 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:41:59.861 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-16T11:41:59.861 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:41:59.932 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rados 2024-09-16T11:41:59.961 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T11:41:59.961 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:59.961 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:41:59.961 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:41:59.961 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T11:41:59.961 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T11:41:59.961 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T11:41:59.961 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T11:41:59.962 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T11:41:59.962 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:41:59.962 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:00.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-fuse 2024-09-16T11:42:00.498 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:42:00.589 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:42:00.632 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:42:00.633 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-16T11:42:00.633 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:42:00.710 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rgw 2024-09-16T11:42:00.782 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:23 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:00.913 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:00.914 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:00.914 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T11:42:00.914 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:00.914 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:42:00.914 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 813 k 2024-09-16T11:42:00.914 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T11:42:00.914 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T11:42:00.914 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:00.914 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:42:00.914 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:00.914 INFO:teuthology.orchestra.run.smithi070.stdout:Install 2 Packages 2024-09-16T11:42:00.914 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:00.915 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 893 k 2024-09-16T11:42:00.915 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 2.6 M 2024-09-16T11:42:00.915 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:42:01.095 INFO:teuthology.orchestra.run.smithi070.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 638 kB/s | 80 kB 00:00 2024-09-16T11:42:01.262 INFO:teuthology.orchestra.run.smithi070.stdout:(2/2): ceph-fuse-17.2.7-1643.gd75f2382.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-09-16T11:42:01.262 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:42:01.263 INFO:teuthology.orchestra.run.smithi070.stdout:Total 2.5 MB/s | 893 kB 00:00 2024-09-16T11:42:01.272 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:42:01.274 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:42:01.296 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:42:01.296 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:42:01.362 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:42:01.374 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:42:01.374 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:42:01.405 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:42:01.406 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-16T11:42:01.406 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:42:01.473 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-cephfs 2024-09-16T11:42:01.555 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:42:01.702 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T11:42:01.778 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-16T11:42:02.041 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:36 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:42:02.131 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:42:02.173 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:42:02.174 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-16T11:42:02.174 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:42:02.249 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rbd 2024-09-16T11:42:02.470 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-16T11:42:02.470 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/2 2024-09-16T11:42:02.729 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T11:42:02.729 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:02.729 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:42:02.729 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T11:42:02.729 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:02.730 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:02.815 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:36 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:42:02.906 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:42:02.948 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:42:02.949 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-16T11:42:02.949 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:42:03.025 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-fuse 2024-09-16T11:42:03.056 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-volume 2024-09-16T11:42:03.597 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:37 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:42:03.636 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:26 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:03.731 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:42:03.732 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:42:03.732 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-09-16T11:42:03.732 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:42:03.732 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:42:03.732 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 82 k 2024-09-16T11:42:03.732 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:03.732 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:42:03.732 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:42:03.732 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-16T11:42:03.732 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:03.733 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 82 k 2024-09-16T11:42:03.733 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 222 k 2024-09-16T11:42:03.733 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:42:03.775 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:03.776 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:03.776 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T11:42:03.776 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:03.776 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:42:03.776 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 270 k 2024-09-16T11:42:03.776 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:03.776 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:42:03.776 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:03.776 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T11:42:03.776 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:03.777 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 270 k 2024-09-16T11:42:03.777 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 1.3 M 2024-09-16T11:42:03.777 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:42:03.930 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-fuse-17.2.7-1643.gd75f2382.el9.x86_64.rpm 417 kB/s | 82 kB 00:00 2024-09-16T11:42:03.930 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:42:03.930 INFO:teuthology.orchestra.run.smithi192.stdout:Total 414 kB/s | 82 kB 00:00 2024-09-16T11:42:03.931 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:42:03.937 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:42:03.937 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:42:03.999 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:42:04.000 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:42:04.000 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-volume-17.2.7-1643.gd75f2382.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-16T11:42:04.001 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:42:04.001 INFO:teuthology.orchestra.run.smithi070.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-16T11:42:04.001 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:42:04.009 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:42:04.010 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:42:04.042 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:42:04.043 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:42:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:42:04.249 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:42:04.276 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:04.310 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:42:04.326 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:42:04.326 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:42:04.326 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T11:42:04.327 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:04.655 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:04.901 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:42:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:42:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:04.901 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:42:05.139 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-mirror 2024-09-16T11:42:05.313 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:42:05.313 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:05.313 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:42:05.313 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:42:05.313 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:05.313 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:05.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados-devel 2024-09-16T11:42:05.708 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:39 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:42:05.841 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:42:05.842 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:42:05.842 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-09-16T11:42:05.842 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:42:05.842 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:42:05.842 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-16T11:42:05.842 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:05.842 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:42:05.842 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:42:05.842 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-16T11:42:05.842 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:05.843 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 2.9 M 2024-09-16T11:42:05.843 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 12 M 2024-09-16T11:42:05.843 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:42:06.157 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-mirror-17.2.7-1643.gd75f2382.el9.x86_64.rpm 9.3 MB/s | 2.9 MB 00:00 2024-09-16T11:42:06.158 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:42:06.158 INFO:teuthology.orchestra.run.smithi192.stdout:Total 9.3 MB/s | 2.9 MB 00:00 2024-09-16T11:42:06.158 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:42:06.167 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:42:06.167 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:42:06.246 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:42:06.246 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:42:06.248 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:29 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:06.382 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:06.383 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:06.383 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-16T11:42:06.383 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:06.383 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:42:06.383 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 126 k 2024-09-16T11:42:06.383 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:06.383 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:42:06.383 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:06.383 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T11:42:06.383 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:06.383 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 126 k 2024-09-16T11:42:06.383 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 448 k 2024-09-16T11:42:06.384 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:42:06.589 INFO:teuthology.orchestra.run.smithi070.stdout:librados-devel-17.2.7-1643.gd75f2382.el9.x86_64 615 kB/s | 126 kB 00:00 2024-09-16T11:42:06.589 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:42:06.590 INFO:teuthology.orchestra.run.smithi070.stdout:Total 611 kB/s | 126 kB 00:00 2024-09-16T11:42:06.590 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:42:06.595 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:42:06.595 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:42:06.657 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:42:06.660 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:42:06.660 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:42:06.717 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:06.742 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:06.742 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:42:06.742 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T11:42:06.742 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T11:42:06.742 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T11:42:06.742 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:06.815 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:42:06.919 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:07.362 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:07.620 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:07.620 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:07.620 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:42:07.621 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:42:07.621 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:07.621 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:07.827 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:07.827 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:07.828 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:42:07.828 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:42:07.828 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:07.828 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:42:07.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs2 2024-09-16T11:42:08.081 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-nbd 2024-09-16T11:42:08.414 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:08.504 INFO:teuthology.orchestra.run.smithi070.stdout:Package libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:42:08.548 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:08.549 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T11:42:08.549 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:08.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs-devel 2024-09-16T11:42:08.663 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:42 ago on Mon 16 Sep 2024 11:40:26 AM UTC. 2024-09-16T11:42:08.801 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-16T11:42:08.802 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:42:08.802 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-09-16T11:42:08.802 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:42:08.802 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-16T11:42:08.802 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 165 k 2024-09-16T11:42:08.802 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:08.802 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-16T11:42:08.802 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-16T11:42:08.802 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-16T11:42:08.803 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:08.803 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 165 k 2024-09-16T11:42:08.803 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 477 k 2024-09-16T11:42:08.803 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-16T11:42:09.019 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-nbd-17.2.7-1643.gd75f2382.el9.x86_64.rpm 765 kB/s | 165 kB 00:00 2024-09-16T11:42:09.019 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:42:09.020 INFO:teuthology.orchestra.run.smithi192.stdout:Total 759 kB/s | 165 kB 00:00 2024-09-16T11:42:09.020 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-16T11:42:09.027 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-16T11:42:09.027 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-16T11:42:09.090 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-16T11:42:09.090 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-16T11:42:09.180 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-16T11:42:09.312 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:09.313 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-16T11:42:09.313 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:09.313 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:42:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-16T11:42:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:09.313 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:42:09.313 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:09.313 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T11:42:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:09.314 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 25 k 2024-09-16T11:42:09.314 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 110 k 2024-09-16T11:42:09.314 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:42:09.323 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:09.468 INFO:teuthology.orchestra.run.smithi070.stdout:libcephfs-devel-17.2.7-1643.gd75f2382.el9.x86_6 162 kB/s | 25 kB 00:00 2024-09-16T11:42:09.469 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:42:09.469 INFO:teuthology.orchestra.run.smithi070.stdout:Total 160 kB/s | 25 kB 00:00 2024-09-16T11:42:09.469 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:42:09.473 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:42:09.473 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:42:09.474 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:42:09.475 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:42:09.550 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:42:09.665 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:09.745 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:10.004 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-16T11:42:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:42:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:10.004 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-16T11:42:10.023 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:10.251 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:10.251 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:10.252 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:42:10.252 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:42:10.252 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:10.252 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:10.282 DEBUG:teuthology.parallel:result is None 2024-09-16T11:42:10.456 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados2 2024-09-16T11:42:11.015 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:11.105 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:42:11.147 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:11.148 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T11:42:11.149 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:11.220 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librbd1 2024-09-16T11:42:11.781 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:11.872 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:42:11.915 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:11.916 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T11:42:11.916 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:11.987 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rados 2024-09-16T11:42:12.548 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:12.637 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:42:12.679 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:12.680 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T11:42:12.680 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:12.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rgw 2024-09-16T11:42:13.318 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:36 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:13.407 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:42:13.449 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:13.450 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T11:42:13.450 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:13.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-cephfs 2024-09-16T11:42:14.085 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:37 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:14.175 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:42:14.217 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:14.218 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T11:42:14.218 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:14.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rbd 2024-09-16T11:42:14.849 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:37 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:14.939 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:42:14.982 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:14.983 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T11:42:14.983 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:15.056 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-fuse 2024-09-16T11:42:15.620 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:38 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:15.752 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:15.753 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:15.753 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-16T11:42:15.753 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:15.753 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:42:15.753 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 82 k 2024-09-16T11:42:15.753 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:15.753 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:42:15.753 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:15.753 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T11:42:15.754 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:15.754 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 82 k 2024-09-16T11:42:15.754 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 222 k 2024-09-16T11:42:15.754 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:42:15.937 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-fuse-17.2.7-1643.gd75f2382.el9.x86_64.rpm 446 kB/s | 82 kB 00:00 2024-09-16T11:42:15.938 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:42:15.938 INFO:teuthology.orchestra.run.smithi070.stdout:Total 442 kB/s | 82 kB 00:00 2024-09-16T11:42:15.938 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:42:15.945 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:42:15.945 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:42:16.008 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:42:16.008 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:42:16.155 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:42:16.250 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:16.671 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:16.933 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:16.933 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:16.933 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:42:16.934 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:42:16.934 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:16.934 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:17.230 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-mirror 2024-09-16T11:42:17.792 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:40 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:17.930 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:17.930 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:17.930 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-16T11:42:17.930 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:17.931 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:42:17.931 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-16T11:42:17.931 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:17.931 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:42:17.931 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:17.931 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T11:42:17.931 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:17.932 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 2.9 M 2024-09-16T11:42:17.932 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 12 M 2024-09-16T11:42:17.932 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:42:18.275 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-mirror-17.2.7-1643.gd75f2382.el9.x86_64.rpm 8.6 MB/s | 2.9 MB 00:00 2024-09-16T11:42:18.275 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:42:18.276 INFO:teuthology.orchestra.run.smithi070.stdout:Total 8.5 MB/s | 2.9 MB 00:00 2024-09-16T11:42:18.276 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:42:18.285 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:42:18.285 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:42:18.363 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:42:18.364 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:42:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:42:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:18.867 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:42:18.867 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T11:42:18.867 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T11:42:18.867 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T11:42:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:19.926 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:19.926 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:19.926 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:42:19.926 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:42:19.926 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:19.926 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:20.178 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-nbd 2024-09-16T11:42:20.754 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:43 ago on Mon 16 Sep 2024 11:40:37 AM UTC. 2024-09-16T11:42:20.891 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T11:42:20.891 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:20.891 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-16T11:42:20.891 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:20.891 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T11:42:20.891 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 165 k 2024-09-16T11:42:20.892 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:20.892 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T11:42:20.892 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T11:42:20.892 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T11:42:20.892 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:20.892 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 165 k 2024-09-16T11:42:20.892 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 477 k 2024-09-16T11:42:20.892 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T11:42:21.095 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-nbd-17.2.7-1643.gd75f2382.el9.x86_64.rpm 817 kB/s | 165 kB 00:00 2024-09-16T11:42:21.096 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:42:21.096 INFO:teuthology.orchestra.run.smithi070.stdout:Total 811 kB/s | 165 kB 00:00 2024-09-16T11:42:21.096 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T11:42:21.103 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T11:42:21.103 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T11:42:21.166 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T11:42:21.166 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T11:42:21.320 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T11:42:21.417 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:21.855 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:22.106 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:42:22.106 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:22.106 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T11:42:22.106 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:42:22.107 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:22.107 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T11:42:22.334 DEBUG:teuthology.parallel:result is None 2024-09-16T11:42:22.335 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T11:42:22.493 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T11:42:22.514 INFO:teuthology.orchestra.run.smithi070.stdout:17.2.7-1643.gd75f2382.el9 2024-09-16T11:42:22.515 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643.gd75f2382.el9 2024-09-16T11:42:22.515 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643.gd75f2382 is installed. 2024-09-16T11:42:22.517 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T11:42:22.668 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T11:42:22.693 INFO:teuthology.orchestra.run.smithi192.stdout:17.2.7-1643.gd75f2382.el9 2024-09-16T11:42:22.694 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643.gd75f2382.el9 2024-09-16T11:42:22.694 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643.gd75f2382 is installed. 2024-09-16T11:42:22.695 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T11:42:22.696 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:22.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T11:42:22.723 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:42:22.723 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T11:42:22.765 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T11:42:22.765 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:22.765 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T11:42:22.791 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T11:42:22.857 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:42:22.857 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T11:42:22.887 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T11:42:22.961 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T11:42:22.961 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:22.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T11:42:22.987 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T11:42:23.053 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:42:23.053 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T11:42:23.083 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T11:42:23.151 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T11:42:23.152 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:23.152 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T11:42:23.176 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T11:42:23.241 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:42:23.241 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T11:42:23.269 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T11:42:23.336 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T11:42:23.423 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T11:42:23.424 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /var/log/ceph 2024-09-16T11:42:23.426 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /var/log/ceph 2024-09-16T11:42:23.452 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T11:42:23.452 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T11:42:23.491 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T11:42:23.518 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T11:42:23.518 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T11:42:23.556 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T11:42:23.593 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T11:42:23.593 INFO:tasks.ceph:config {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T11:42:23.593 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907206', 'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/cache thrashosds-health workloads/rbd_fsx_rate_limit}', 'email': 'sangadi@redhat.com', 'first_in_suite': False, 'job_id': '7907206', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'owner': 'scheduled_sangadi@teuthology', 'priority': 130, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 9764, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rbd', 'suite_branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'targets': {'smithi070.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPc++QQ1w+1BamHjutN/d3o3/GMdv+Qh4ZfhfnaWNiAGwRtQz0dM7dnkMQLsozwhCkp0Cd1JQ11z2aRmFeNM6bM=', 'smithi192.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCCNyl/oqltRRr7bcKNOoN8heGT9kiGQBHhiGOt499Hn8rod5PGOXPfDIqACF6JXQmU7sBL78Fk+49X2VBm6t0o='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'exec': {'client.0': ['sudo ceph osd pool create cache 4', 'sudo ceph osd tier add rbd cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay rbd cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 60', 'sudo ceph osd pool set cache target_max_objects 250']}}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-16_10:44:09', 'tube': 'smithi', 'user': 'sangadi', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T11:42:23.594 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T11:42:23.621 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T11:42:23.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T11:42:23.677 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T11:42:23.706 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:23.706 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T11:42:23.762 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T11:42:23.762 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2024-09-16T11:42:23.817 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T11:42:23.818 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:42:23.818 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-16T11:42:23.818 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:42:23.818 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T11:42:23.818 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-16 11:42:19.105073114 +0000 2024-09-16T11:42:23.818 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-16 11:39:46.387924441 +0000 2024-09-16T11:42:23.818 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-16 11:39:46.387924441 +0000 2024-09-16T11:42:23.818 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-16 11:39:46.387924441 +0000 2024-09-16T11:42:23.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T11:42:23.881 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-16T11:42:23.882 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-16T11:42:23.882 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000103588 s, 4.9 MB/s 2024-09-16T11:42:23.883 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T11:42:23.939 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2024-09-16T11:42:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T11:42:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:42:23.994 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T11:42:23.994 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:42:23.994 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T11:42:23.994 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-16 11:42:19.106073084 +0000 2024-09-16T11:42:23.994 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-16 11:39:46.712914358 +0000 2024-09-16T11:42:23.994 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-16 11:39:46.712914358 +0000 2024-09-16T11:42:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-16 11:39:46.712914358 +0000 2024-09-16T11:42:23.995 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T11:42:24.057 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-16T11:42:24.057 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-16T11:42:24.057 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000114785 s, 4.5 MB/s 2024-09-16T11:42:24.058 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T11:42:24.114 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2024-09-16T11:42:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T11:42:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:42:24.169 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-16T11:42:24.170 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:42:24.170 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T11:42:24.170 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-16 11:42:19.106073084 +0000 2024-09-16T11:42:24.170 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-16 11:39:47.031904462 +0000 2024-09-16T11:42:24.170 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-16 11:39:47.031904462 +0000 2024-09-16T11:42:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-16 11:39:47.031904462 +0000 2024-09-16T11:42:24.170 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T11:42:24.232 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-16T11:42:24.232 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-16T11:42:24.232 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 9.651e-05 s, 5.3 MB/s 2024-09-16T11:42:24.233 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T11:42:24.289 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2024-09-16T11:42:24.344 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T11:42:24.344 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:42:24.344 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-16T11:42:24.345 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:42:24.345 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T11:42:24.345 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-16 11:42:19.107073054 +0000 2024-09-16T11:42:24.345 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-16 11:39:47.350894565 +0000 2024-09-16T11:42:24.345 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-16 11:39:47.350894565 +0000 2024-09-16T11:42:24.345 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-16 11:39:47.350894565 +0000 2024-09-16T11:42:24.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T11:42:24.407 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-16T11:42:24.407 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-16T11:42:24.407 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000106274 s, 4.8 MB/s 2024-09-16T11:42:24.409 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T11:42:24.465 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T11:42:24.465 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:42:24.465 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T11:42:24.483 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T11:42:24.483 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-09-16T11:42:24.540 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T11:42:24.541 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:42:24.541 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T11:42:24.541 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:42:24.541 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T11:42:24.541 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-16 11:42:06.982051510 +0000 2024-09-16T11:42:24.541 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-16 11:39:46.406329224 +0000 2024-09-16T11:42:24.541 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-16 11:39:46.406329224 +0000 2024-09-16T11:42:24.541 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-16 11:39:46.406329224 +0000 2024-09-16T11:42:24.541 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T11:42:24.606 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-16T11:42:24.606 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-16T11:42:24.606 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000336118 s, 1.5 MB/s 2024-09-16T11:42:24.608 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T11:42:24.666 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-09-16T11:42:24.724 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T11:42:24.724 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:42:24.724 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-16T11:42:24.724 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:42:24.724 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T11:42:24.724 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-16 11:42:06.983051490 +0000 2024-09-16T11:42:24.725 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-16 11:39:46.729322807 +0000 2024-09-16T11:42:24.725 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-16 11:39:46.729322807 +0000 2024-09-16T11:42:24.725 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-16 11:39:46.729322807 +0000 2024-09-16T11:42:24.725 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T11:42:24.789 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-16T11:42:24.789 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-16T11:42:24.789 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000199798 s, 2.6 MB/s 2024-09-16T11:42:24.790 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T11:42:24.847 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-09-16T11:42:24.903 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T11:42:24.904 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:42:24.904 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-16T11:42:24.904 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:42:24.904 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T11:42:24.904 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-16 11:42:06.983051490 +0000 2024-09-16T11:42:24.904 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-16 11:39:47.057316291 +0000 2024-09-16T11:42:24.904 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-16 11:39:47.057316291 +0000 2024-09-16T11:42:24.904 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-16 11:39:47.057316291 +0000 2024-09-16T11:42:24.904 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T11:42:24.967 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-16T11:42:24.968 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-16T11:42:24.968 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000199014 s, 2.6 MB/s 2024-09-16T11:42:24.969 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T11:42:25.026 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-09-16T11:42:25.082 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T11:42:25.082 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:42:25.082 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-16T11:42:25.082 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:42:25.082 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T11:42:25.082 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-16 11:42:06.984051471 +0000 2024-09-16T11:42:25.082 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-16 11:39:47.365310172 +0000 2024-09-16T11:42:25.082 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-16 11:39:47.365310172 +0000 2024-09-16T11:42:25.082 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-16 11:39:47.365310172 +0000 2024-09-16T11:42:25.083 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T11:42:25.146 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-16T11:42:25.146 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-16T11:42:25.146 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000252937 s, 2.0 MB/s 2024-09-16T11:42:25.147 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T11:42:25.204 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T11:42:25.204 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T11:42:25.204 INFO:tasks.ceph:Generating config... 2024-09-16T11:42:25.208 INFO:tasks.ceph:[client] rbd qos iops burst = 100 2024-09-16T11:42:25.208 INFO:tasks.ceph:[client] rbd qos iops limit = 50 2024-09-16T11:42:25.209 INFO:tasks.ceph:[client] rbd qos schedule tick min = 100 2024-09-16T11:42:25.209 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-16T11:42:25.209 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-16T11:42:25.209 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-16T11:42:25.209 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T11:42:25.209 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T11:42:25.209 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T11:42:25.209 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T11:42:25.209 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T11:42:25.209 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T11:42:25.209 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-09-16T11:42:25.209 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-16T11:42:25.209 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T11:42:25.210 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-16T11:42:25.210 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-16T11:42:25.210 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T11:42:25.210 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T11:42:25.210 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-16T11:42:25.210 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T11:42:25.210 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T11:42:25.211 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T11:42:25.211 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T11:42:25.211 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-16T11:42:25.211 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T11:42:25.211 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-16T11:42:25.211 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T11:42:25.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T11:42:25.258 INFO:teuthology.orchestra.run.smithi070.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T11:42:25.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T11:42:25.301 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T11:42:25.366 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.70'), ('mon.c', '[v2:172.21.15.70:3301,v1:172.21.15.70:6790]'), ('mon.b', '172.21.15.192')] 2024-09-16T11:42:25.367 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790],172.21.15.192', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-16T11:42:25.368 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:25.368 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T11:42:25.423 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.70 --addv c '[v2:172.21.15.70:3301,v1:172.21.15.70:6790]' --add b 172.21.15.192 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:25.504 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: generated fsid 3420bcad-625f-4e15-aaff-fb4236322168 2024-09-16T11:42:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:setting min_mon_release = octopus 2024-09-16T11:42:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 0 2024-09-16T11:42:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:fsid 3420bcad-625f-4e15-aaff-fb4236322168 2024-09-16T11:42:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:last_changed 2024-09-16T11:42:25.503417+0000 2024-09-16T11:42:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-09-16T11:42:25.503417+0000 2024-09-16T11:42:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:min_mon_release 15 (octopus) 2024-09-16T11:42:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:election_strategy: 1 2024-09-16T11:42:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.a 2024-09-16T11:42:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.b 2024-09-16T11:42:25.505 INFO:teuthology.orchestra.run.smithi070.stdout:2: [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] mon.c 2024-09-16T11:42:25.506 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T11:42:25.507 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T11:42:25.563 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 3420bcad-625f-4e15-aaff-fb4236322168... 2024-09-16T11:42:25.564 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T11:42:25.605 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T11:42:25.646 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2024-09-16T11:42:25.646 INFO:teuthology.orchestra.run.smithi070.stdout: chdir = "" 2024-09-16T11:42:25.646 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T11:42:25.646 INFO:teuthology.orchestra.run.smithi070.stdout: auth supported = cephx 2024-09-16T11:42:25.646 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.646 INFO:teuthology.orchestra.run.smithi070.stdout: filestore xattr use omap = true 2024-09-16T11:42:25.646 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.646 INFO:teuthology.orchestra.run.smithi070.stdout: mon clock drift allowed = 1.000 2024-09-16T11:42:25.646 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.646 INFO:teuthology.orchestra.run.smithi070.stdout: osd crush chooseleaf type = 0 2024-09-16T11:42:25.647 INFO:teuthology.orchestra.run.smithi070.stdout: auth debug = true 2024-09-16T11:42:25.647 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.647 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on old message = true 2024-09-16T11:42:25.647 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on bug = true 2024-09-16T11:42:25.647 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.647 INFO:teuthology.orchestra.run.smithi070.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T11:42:25.647 INFO:teuthology.orchestra.run.smithi070.stdout: mon pg warn max object skew = 0 2024-09-16T11:42:25.647 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.647 INFO:teuthology.orchestra.run.smithi070.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T11:42:25.647 INFO:teuthology.orchestra.run.smithi070.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T11:42:25.648 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.648 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default size = 2 2024-09-16T11:42:25.648 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.648 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow primary affinity = true 2024-09-16T11:42:25.648 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow pg remap = true 2024-09-16T11:42:25.648 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on legacy crush tunables = false 2024-09-16T11:42:25.648 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on crush straw calc version zero = false 2024-09-16T11:42:25.648 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on no sortbitwise = false 2024-09-16T11:42:25.648 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on osd down out interval zero = false 2024-09-16T11:42:25.648 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on too few osds = false 2024-09-16T11:42:25.648 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T11:42:25.649 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T11:42:25.649 INFO:teuthology.orchestra.run.smithi070.stdout: mon_allow_pool_size_one = true 2024-09-16T11:42:25.649 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.649 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T11:42:25.649 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.649 INFO:teuthology.orchestra.run.smithi070.stdout: osd default data pool replay window = 5 2024-09-16T11:42:25.649 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.649 INFO:teuthology.orchestra.run.smithi070.stdout: mon allow pool delete = true 2024-09-16T11:42:25.649 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.649 INFO:teuthology.orchestra.run.smithi070.stdout: mon cluster log file level = debug 2024-09-16T11:42:25.649 INFO:teuthology.orchestra.run.smithi070.stdout: debug asserts on shutdown = true 2024-09-16T11:42:25.649 INFO:teuthology.orchestra.run.smithi070.stdout: mon health detail to clog = false 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout: mon host = "172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790],172.21.15.192" 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout: mon client directed command retry = 5 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on pool no app = False 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout: ms inject socket failures = 5000 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = 3420bcad-625f-4e15-aaff-fb4236322168 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout:[osd] 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout: osd journal size = 100 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub load threshold = 5.0 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub max interval = 600 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout: osd mclock profile = high_recovery_ops 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.650 INFO:teuthology.orchestra.run.smithi070.stdout: osd recover clone overlap = true 2024-09-16T11:42:25.651 INFO:teuthology.orchestra.run.smithi070.stdout: osd recovery max chunk = 1048576 2024-09-16T11:42:25.651 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.651 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug shutdown = true 2024-09-16T11:42:25.651 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug op order = true 2024-09-16T11:42:25.651 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug verify stray on activate = true 2024-09-16T11:42:25.651 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.651 INFO:teuthology.orchestra.run.smithi070.stdout: osd open classes on start = true 2024-09-16T11:42:25.651 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug pg log writeout = true 2024-09-16T11:42:25.651 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.651 INFO:teuthology.orchestra.run.smithi070.stdout: osd deep scrub update digest min age = 30 2024-09-16T11:42:25.651 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.651 INFO:teuthology.orchestra.run.smithi070.stdout: osd map max advance = 10 2024-09-16T11:42:25.651 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: journal zero on create = true 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: filestore ondisk finisher threads = 3 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: filestore apply finisher threads = 3 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: bdev debug aio = true 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug misdirected ops = true 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore block size = 96636764160 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore compression algorithm = zstd 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore compression mode = aggressive 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore fsck on mount = True 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluefs = 1/20 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluestore = 1/20 2024-09-16T11:42:25.652 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-16T11:42:25.653 INFO:teuthology.orchestra.run.smithi070.stdout: debug osd = 20 2024-09-16T11:42:25.653 INFO:teuthology.orchestra.run.smithi070.stdout: debug rocksdb = 4/10 2024-09-16T11:42:25.653 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T11:42:25.653 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd full ratio = 0.9 2024-09-16T11:42:25.653 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd nearfull ratio = 0.8 2024-09-16T11:42:25.653 INFO:teuthology.orchestra.run.smithi070.stdout: osd failsafe full ratio = 0.95 2024-09-16T11:42:25.653 INFO:teuthology.orchestra.run.smithi070.stdout: osd max markdown count = 1000 2024-09-16T11:42:25.653 INFO:teuthology.orchestra.run.smithi070.stdout: osd objectstore = bluestore 2024-09-16T11:42:25.653 INFO:teuthology.orchestra.run.smithi070.stdout: osd shutdown pgref assert = True 2024-09-16T11:42:25.653 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.653 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr] 2024-09-16T11:42:25.653 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-16T11:42:25.653 INFO:teuthology.orchestra.run.smithi070.stdout: debug mgr = 20 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout: mgr/telemetry/nag = false 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout:[mon] 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout: debug paxos = 20 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout: mon data avail warn = 5 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout: mon mgr mkfs grace = 240 2024-09-16T11:42:25.654 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd reporter subtree level = osd 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd prime pg temp = true 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: # 1m isn't quite enough 2024-09-16T11:42:25.655 INFO:teuthology.orchestra.run.smithi070.stdout: mon_down_mkfs_grace = 2m 2024-09-16T11:42:25.656 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.656 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_filestore_osds = false 2024-09-16T11:42:25.656 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:42:25.656 INFO:teuthology.orchestra.run.smithi070.stdout:[client] 2024-09-16T11:42:25.656 INFO:teuthology.orchestra.run.smithi070.stdout: rgw cache enabled = true 2024-09-16T11:42:25.656 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable ops log = true 2024-09-16T11:42:25.656 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable usage log = true 2024-09-16T11:42:25.656 INFO:teuthology.orchestra.run.smithi070.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T11:42:25.656 INFO:teuthology.orchestra.run.smithi070.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T11:42:25.656 INFO:teuthology.orchestra.run.smithi070.stdout: rbd qos iops burst = 100 2024-09-16T11:42:25.656 INFO:teuthology.orchestra.run.smithi070.stdout: rbd qos iops limit = 50 2024-09-16T11:42:25.656 INFO:teuthology.orchestra.run.smithi070.stdout: rbd qos schedule tick min = 100 2024-09-16T11:42:25.656 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.a] 2024-09-16T11:42:25.657 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.c] 2024-09-16T11:42:25.657 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.b] 2024-09-16T11:42:25.657 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2024-09-16T11:42:25.657 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2024-09-16T11:42:25.658 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T11:42:25.658 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2024-09-16T11:42:25.658 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.658 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2024-09-16T11:42:25.658 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.658 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2024-09-16T11:42:25.658 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.658 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2024-09-16T11:42:25.658 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2024-09-16T11:42:25.658 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.658 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2024-09-16T11:42:25.659 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2024-09-16T11:42:25.660 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2024-09-16T11:42:25.660 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2024-09-16T11:42:25.660 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2024-09-16T11:42:25.660 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T11:42:25.660 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T11:42:25.660 INFO:teuthology.orchestra.run.smithi192.stdout: mon_allow_pool_size_one = true 2024-09-16T11:42:25.660 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.660 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T11:42:25.660 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.660 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2024-09-16T11:42:25.660 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.660 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2024-09-16T11:42:25.660 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: mon health detail to clog = false 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790],172.21.15.192" 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: mon client directed command retry = 5 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on pool no app = False 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: ms inject socket failures = 5000 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = 3420bcad-625f-4e15-aaff-fb4236322168 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2024-09-16T11:42:25.661 INFO:teuthology.orchestra.run.smithi192.stdout: osd mclock profile = high_recovery_ops 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.662 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2024-09-16T11:42:25.663 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.663 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2024-09-16T11:42:25.663 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.663 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2024-09-16T11:42:25.663 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2024-09-16T11:42:25.663 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.663 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2024-09-16T11:42:25.663 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2024-09-16T11:42:25.663 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore block size = 96636764160 2024-09-16T11:42:25.664 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore compression algorithm = zstd 2024-09-16T11:42:25.664 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore compression mode = aggressive 2024-09-16T11:42:25.664 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore fsck on mount = True 2024-09-16T11:42:25.664 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluefs = 1/20 2024-09-16T11:42:25.664 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluestore = 1/20 2024-09-16T11:42:25.664 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-09-16T11:42:25.664 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 20 2024-09-16T11:42:25.664 INFO:teuthology.orchestra.run.smithi192.stdout: debug rocksdb = 4/10 2024-09-16T11:42:25.665 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T11:42:25.665 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd full ratio = 0.9 2024-09-16T11:42:25.665 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd nearfull ratio = 0.8 2024-09-16T11:42:25.665 INFO:teuthology.orchestra.run.smithi192.stdout: osd failsafe full ratio = 0.95 2024-09-16T11:42:25.665 INFO:teuthology.orchestra.run.smithi192.stdout: osd max markdown count = 1000 2024-09-16T11:42:25.665 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = bluestore 2024-09-16T11:42:25.666 INFO:teuthology.orchestra.run.smithi192.stdout: osd shutdown pgref assert = True 2024-09-16T11:42:25.666 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.666 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2024-09-16T11:42:25.666 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-09-16T11:42:25.666 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2024-09-16T11:42:25.666 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-09-16T11:42:25.666 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-09-16T11:42:25.667 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-09-16T11:42:25.667 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-09-16T11:42:25.667 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2024-09-16T11:42:25.667 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.667 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2024-09-16T11:42:25.667 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-09-16T11:42:25.667 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-09-16T11:42:25.668 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2024-09-16T11:42:25.668 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-09-16T11:42:25.668 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2024-09-16T11:42:25.668 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2024-09-16T11:42:25.668 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-09-16T11:42:25.668 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2024-09-16T11:42:25.668 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2024-09-16T11:42:25.668 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-09-16T11:42:25.669 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.669 INFO:teuthology.orchestra.run.smithi192.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T11:42:25.669 INFO:teuthology.orchestra.run.smithi192.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T11:42:25.669 INFO:teuthology.orchestra.run.smithi192.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T11:42:25.669 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.669 INFO:teuthology.orchestra.run.smithi192.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T11:42:25.669 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T11:42:25.669 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T11:42:25.670 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.670 INFO:teuthology.orchestra.run.smithi192.stdout: # 1m isn't quite enough 2024-09-16T11:42:25.670 INFO:teuthology.orchestra.run.smithi192.stdout: mon_down_mkfs_grace = 2m 2024-09-16T11:42:25.670 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.670 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_filestore_osds = false 2024-09-16T11:42:25.670 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-16T11:42:25.670 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2024-09-16T11:42:25.670 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2024-09-16T11:42:25.670 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2024-09-16T11:42:25.671 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2024-09-16T11:42:25.671 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T11:42:25.671 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T11:42:25.671 INFO:teuthology.orchestra.run.smithi192.stdout: rbd qos iops burst = 100 2024-09-16T11:42:25.671 INFO:teuthology.orchestra.run.smithi192.stdout: rbd qos iops limit = 50 2024-09-16T11:42:25.671 INFO:teuthology.orchestra.run.smithi192.stdout: rbd qos schedule tick min = 100 2024-09-16T11:42:25.671 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2024-09-16T11:42:25.671 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2024-09-16T11:42:25.671 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2024-09-16T11:42:25.673 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T11:42:25.674 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T11:42:25.762 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T11:42:25.763 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:25.763 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T11:42:25.778 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:25.778 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T11:42:25.835 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T11:42:25.835 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:25.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T11:42:25.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T11:42:25.910 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:25.910 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:25.965 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi192.front.sepia.ceph.com 2024-09-16T11:42:25.965 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:42:25.966 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T11:42:25.966 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T11:42:26.000 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:42:26.000 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:26.056 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T11:42:26.056 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T11:42:26.056 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T11:42:26.103 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T11:42:26.105 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T11:42:26.205 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T11:42:26.207 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T11:42:26.208 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T11:42:26.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T11:42:26.246 INFO:teuthology.orchestra.run.smithi070.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T11:42:26.258 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T11:42:26.295 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-16T11:42:26.307 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T11:42:26.308 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T11:42:26.308 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T11:42:26.332 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T11:42:26.332 INFO:tasks.ceph:role: osd.0 2024-09-16T11:42:26.332 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T11:42:26.332 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T11:42:26.396 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:42:26.396 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:42:26.396 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:42:26.396 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T11:42:26.397 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:42:26.397 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:42:26.397 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:42:26.397 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T11:42:26.397 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:42:26.397 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:42:26.429 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-16T11:42:26.431 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-16T11:42:26.432 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T11:42:26.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T11:42:26.570 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T11:42:26.635 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T11:42:26.635 INFO:tasks.ceph:role: osd.1 2024-09-16T11:42:26.636 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T11:42:26.636 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T11:42:26.700 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:42:26.700 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:42:26.701 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:42:26.701 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T11:42:26.701 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:42:26.701 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:42:26.701 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:42:26.701 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T11:42:26.701 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:42:26.701 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:42:26.733 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-16T11:42:26.734 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-16T11:42:26.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T11:42:26.803 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T11:42:26.876 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T11:42:26.942 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T11:42:26.943 INFO:tasks.ceph:role: osd.2 2024-09-16T11:42:26.943 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T11:42:26.943 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T11:42:27.008 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:42:27.008 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:42:27.008 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:42:27.009 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T11:42:27.009 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:42:27.009 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:42:27.009 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:42:27.009 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T11:42:27.009 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:42:27.009 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:42:27.038 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-16T11:42:27.040 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-16T11:42:27.040 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T11:42:27.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T11:42:27.177 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T11:42:27.241 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T11:42:27.242 INFO:tasks.ceph:role: osd.3 2024-09-16T11:42:27.242 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T11:42:27.242 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T11:42:27.306 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:42:27.306 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:42:27.306 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:42:27.306 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T11:42:27.306 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:42:27.306 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:42:27.306 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:42:27.306 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T11:42:27.306 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:42:27.306 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:42:27.338 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-16T11:42:27.339 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-16T11:42:27.340 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-16T11:42:27.408 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T11:42:27.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:27.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:27.586+0000 7f2adf520440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T11:42:27.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:27.586+0000 7f2adf520440 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T11:42:27.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:27.586+0000 7f2adf520440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T11:42:27.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:27.586+0000 7f2adf520440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T11:42:27.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:27.586+0000 7f2adf520440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T11:42:27.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:27.586+0000 7f2adf520440 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T11:42:28.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:28.112+0000 7f2adf520440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:28.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:28.878+0000 7f2adf520440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:29.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:29.640+0000 7f2adf520440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:30.437 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T11:42:30.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:30.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:30.549+0000 7ff121c3d440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T11:42:30.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:30.550+0000 7ff121c3d440 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T11:42:30.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:30.550+0000 7ff121c3d440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T11:42:30.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:30.550+0000 7ff121c3d440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T11:42:30.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:30.550+0000 7ff121c3d440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T11:42:30.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:30.550+0000 7ff121c3d440 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T11:42:31.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:31.073+0000 7ff121c3d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:31.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:31.839+0000 7ff121c3d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:32.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:32.601+0000 7ff121c3d440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:33.408 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T11:42:33.438 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:33.522+0000 7f31b5633440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T11:42:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:33.522+0000 7f31b5633440 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T11:42:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:33.522+0000 7f31b5633440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T11:42:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:33.522+0000 7f31b5633440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T11:42:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:33.522+0000 7f31b5633440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T11:42:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:33.523+0000 7f31b5633440 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T11:42:34.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:34.049+0000 7f31b5633440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:34.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:34.810+0000 7f31b5633440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:35.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:35.573+0000 7f31b5633440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:36.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T11:42:36.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:36.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:36.484+0000 7f89dc60c440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T11:42:36.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:36.484+0000 7f89dc60c440 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T11:42:36.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:36.484+0000 7f89dc60c440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T11:42:36.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:36.484+0000 7f89dc60c440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T11:42:36.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:36.484+0000 7f89dc60c440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T11:42:36.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:36.484+0000 7f89dc60c440 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T11:42:37.011 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:37.009+0000 7f89dc60c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:37.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:37.772+0000 7f89dc60c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:38.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:42:38.535+0000 7f89dc60c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:39.332 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T11:42:39.362 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-16T11:42:39.392 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T11:42:39.392 INFO:tasks.ceph:role: osd.4 2024-09-16T11:42:39.392 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-16T11:42:39.392 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T11:42:39.469 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:42:39.469 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:42:39.469 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:42:39.470 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T11:42:39.470 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:42:39.470 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:42:39.470 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:42:39.470 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T11:42:39.470 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:42:39.470 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:42:39.976 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-09-16T11:42:39.978 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-09-16T11:42:39.978 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-16T11:42:40.079 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-16T11:42:40.111 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-16T11:42:40.177 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T11:42:40.177 INFO:tasks.ceph:role: osd.5 2024-09-16T11:42:40.177 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-16T11:42:40.177 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T11:42:40.251 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:42:40.251 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:42:40.251 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:42:40.251 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T11:42:40.251 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:42:40.251 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:42:40.251 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:42:40.251 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T11:42:40.251 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:42:40.251 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:42:40.721 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-09-16T11:42:40.722 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-09-16T11:42:40.723 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-16T11:42:40.803 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-16T11:42:40.872 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-16T11:42:40.936 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T11:42:40.936 INFO:tasks.ceph:role: osd.6 2024-09-16T11:42:40.936 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-16T11:42:40.937 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T11:42:41.008 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:42:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:42:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:42:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T11:42:41.009 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:42:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:42:41.009 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:42:41.009 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T11:42:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:42:41.009 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:42:41.542 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-09-16T11:42:41.544 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-09-16T11:42:41.545 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-16T11:42:41.690 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-16T11:42:41.719 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-16T11:42:41.785 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T11:42:41.785 INFO:tasks.ceph:role: osd.7 2024-09-16T11:42:41.785 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-16T11:42:41.785 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T11:42:41.857 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:42:41.858 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:42:41.858 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:42:41.858 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T11:42:41.858 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:42:41.858 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:42:41.858 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:42:41.858 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T11:42:41.858 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:42:41.858 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:42:42.368 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-09-16T11:42:42.371 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-09-16T11:42:42.371 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-16T11:42:42.452 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-16T11:42:42.523 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:42.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:42.622+0000 7f3909811440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-16T11:42:42.625 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:42.623+0000 7f3909811440 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-16T11:42:42.625 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:42.623+0000 7f3909811440 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T11:42:42.625 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:42.623+0000 7f3909811440 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T11:42:42.625 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:42.623+0000 7f3909811440 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T11:42:42.625 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:42.623+0000 7f3909811440 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-16T11:42:43.153 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:43.151+0000 7f3909811440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:43.918 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:43.917+0000 7f3909811440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:44.681 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:44.679+0000 7f3909811440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:45.474 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T11:42:45.500 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:45.583 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:45.581+0000 7f2fb5826440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-16T11:42:45.583 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:45.581+0000 7f2fb5826440 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-16T11:42:45.583 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:45.581+0000 7f2fb5826440 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T11:42:45.584 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:45.582+0000 7f2fb5826440 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T11:42:45.584 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:45.582+0000 7f2fb5826440 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T11:42:45.584 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:45.582+0000 7f2fb5826440 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-16T11:42:46.107 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:46.105+0000 7f2fb5826440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:46.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:46.872+0000 7f2fb5826440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:47.636 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:47.635+0000 7f2fb5826440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:48.434 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T11:42:48.504 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:48.591 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:48.589+0000 7f8b06f2c440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-16T11:42:48.591 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:48.589+0000 7f8b06f2c440 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-16T11:42:48.591 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:48.589+0000 7f8b06f2c440 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T11:42:48.591 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:48.589+0000 7f8b06f2c440 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T11:42:48.591 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:48.589+0000 7f8b06f2c440 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T11:42:48.591 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:48.590+0000 7f8b06f2c440 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-16T11:42:49.115 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:49.114+0000 7f8b06f2c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:49.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:49.875+0000 7f8b06f2c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:50.641 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:50.639+0000 7f8b06f2c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:51.436 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-16T11:42:51.463 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:51.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:51.545+0000 7f512268b440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-16T11:42:51.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:51.545+0000 7f512268b440 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-16T11:42:51.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:51.545+0000 7f512268b440 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T11:42:51.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:51.546+0000 7f512268b440 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T11:42:51.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:51.546+0000 7f512268b440 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T11:42:51.548 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:51.546+0000 7f512268b440 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-16T11:42:52.073 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:52.072+0000 7f512268b440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:52.840 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:52.838+0000 7f512268b440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:53.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-16T11:42:53.605+0000 7f512268b440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:42:54.404 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-16T11:42:54.474 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T11:42:54.475 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:54.475 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-16T11:42:54.505 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:54.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T11:42:54.575 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:54.575 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T11:42:54.642 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:54.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T11:42:54.708 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:54.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T11:42:54.774 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:42:54.774 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T11:42:54.804 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:42:54.804 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-16T11:42:54.873 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:42:54.873 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-16T11:42:54.942 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:42:54.942 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-16T11:42:55.011 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:42:55.011 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-16T11:42:55.079 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:42:55.079 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T11:42:55.097 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:42:55.097 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-16T11:42:55.136 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T11:42:55.136 DEBUG:teuthology.orchestra.run.smithi070:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T11:42:55.139 DEBUG:teuthology.orchestra.run.smithi192:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T11:42:55.179 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.y] 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCiGehmMHMUBhAAMR1479xte36cJMQxhFm/ng== 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.0] 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCjGehm1OH+IhAAx4qnYbcm/K4+V/lpGNsUsw== 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.1] 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCmGehmrt/WIBAA1G4Ya6bPRFUR60q6QBI7tA== 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.2] 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCpGehm6lA2HxAAKxrS6+/I+jb3pzv6vjV6/g== 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.3] 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCsGehmq93uHBAAh35W8JVGEzfJTka8eVeXAA== 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.x] 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCiGehmkVUtDBAAGeI+L/ODz/GbeivCCYItjQ== 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.4] 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCyGehmcowkJRAAS3Ce0r1DPia7Uuw+i/mbYw== 2024-09-16T11:42:55.180 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.5] 2024-09-16T11:42:55.181 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQC1GehmxfaxIhAAC7KDNEazmjTRHOvb8mIyUg== 2024-09-16T11:42:55.181 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.6] 2024-09-16T11:42:55.181 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQC4Gehm2pAkIxAAz5iCQsz3dgDKKpvrq9YqOA== 2024-09-16T11:42:55.181 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.7] 2024-09-16T11:42:55.181 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQC7GehmCMaLIBAA8OjeACNYSTDUDqRJGOMAzw== 2024-09-16T11:42:55.181 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2024-09-16T11:42:55.181 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCiGehmFTOXDhAAdxJBlgLaePT9u6P8hRBriw== 2024-09-16T11:42:55.181 INFO:teuthology.orchestra.run.smithi070.stdout:[client.1] 2024-09-16T11:42:55.182 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCiGehm0SyQERAAnxnUhT9t8KNK5P6+C+ktsQ== 2024-09-16T11:42:55.201 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.y] 2024-09-16T11:42:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCiGehmMHMUBhAAMR1479xte36cJMQxhFm/ng== 2024-09-16T11:42:55.201 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.0] 2024-09-16T11:42:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCjGehm1OH+IhAAx4qnYbcm/K4+V/lpGNsUsw== 2024-09-16T11:42:55.201 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.1] 2024-09-16T11:42:55.201 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCmGehmrt/WIBAA1G4Ya6bPRFUR60q6QBI7tA== 2024-09-16T11:42:55.202 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.2] 2024-09-16T11:42:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCpGehm6lA2HxAAKxrS6+/I+jb3pzv6vjV6/g== 2024-09-16T11:42:55.202 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.3] 2024-09-16T11:42:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCsGehmq93uHBAAh35W8JVGEzfJTka8eVeXAA== 2024-09-16T11:42:55.202 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.x] 2024-09-16T11:42:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCiGehmkVUtDBAAGeI+L/ODz/GbeivCCYItjQ== 2024-09-16T11:42:55.202 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.4] 2024-09-16T11:42:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCyGehmcowkJRAAS3Ce0r1DPia7Uuw+i/mbYw== 2024-09-16T11:42:55.202 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.5] 2024-09-16T11:42:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQC1GehmxfaxIhAAC7KDNEazmjTRHOvb8mIyUg== 2024-09-16T11:42:55.202 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.6] 2024-09-16T11:42:55.202 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQC4Gehm2pAkIxAAz5iCQsz3dgDKKpvrq9YqOA== 2024-09-16T11:42:55.202 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.7] 2024-09-16T11:42:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQC7GehmCMaLIBAA8OjeACNYSTDUDqRJGOMAzw== 2024-09-16T11:42:55.203 INFO:teuthology.orchestra.run.smithi192.stdout:[client.0] 2024-09-16T11:42:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCiGehmFTOXDhAAdxJBlgLaePT9u6P8hRBriw== 2024-09-16T11:42:55.203 INFO:teuthology.orchestra.run.smithi192.stdout:[client.1] 2024-09-16T11:42:55.203 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCiGehm0SyQERAAnxnUhT9t8KNK5P6+C+ktsQ== 2024-09-16T11:42:55.204 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T11:42:55.225 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T11:42:55.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.295 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.341 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.344 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.392 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.437 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.440 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T11:42:55.488 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T11:42:55.534 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.537 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.584 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.587 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.635 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.681 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.684 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:42:55.729 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T11:42:55.732 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T11:42:55.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T11:42:55.783 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T11:42:55.832 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T11:42:55.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T11:42:55.856 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T11:42:55.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T11:42:55.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T11:42:56.036 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T11:42:56.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T11:42:56.151 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T11:42:56.177 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T11:42:56.270 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T11:42:56.295 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:56.297 DEBUG:teuthology.orchestra.run.smithi192:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:42:56.350 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T11:42:56.351 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T11:42:56.351 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T11:42:56.353 INFO:tasks.ceph.mon.a:Started 2024-09-16T11:42:56.353 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T11:42:56.353 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T11:42:56.355 INFO:tasks.ceph.mon.c:Started 2024-09-16T11:42:56.355 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T11:42:56.355 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T11:42:56.392 INFO:tasks.ceph.mon.b:Started 2024-09-16T11:42:56.393 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T11:42:56.393 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-16T11:42:56.393 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-16T11:42:56.395 INFO:tasks.ceph.mgr.y:Started 2024-09-16T11:42:56.395 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T11:42:56.395 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T11:42:56.397 INFO:tasks.ceph.mgr.x:Started 2024-09-16T11:42:56.397 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T11:42:56.398 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T11:43:01.776 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:01.773+0000 7f4a0522cec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T11:43:01.777 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:01.774+0000 7f05b0e53ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T11:43:01.815 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:01.813+0000 7f4a0522cec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T11:43:01.816 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:01.814+0000 7f05b0e53ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T11:43:01.864 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:01.862+0000 7f4a0522cec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T11:43:01.865 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:01.863+0000 7f05b0e53ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T11:43:02.003 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:02.000+0000 7f4a0522cec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T11:43:02.021 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:02.018+0000 7f05b0e53ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T11:43:02.108 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:02.106+0000 7f4a0522cec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T11:43:02.126 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:02.123+0000 7f05b0e53ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T11:43:02.419 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:02.417+0000 7f05b0e53ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T11:43:02.428 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:02.426+0000 7f4a0522cec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T11:43:02.469 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:02.466+0000 7f05b0e53ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T11:43:02.481 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:02.479+0000 7f4a0522cec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T11:43:02.569 INFO:teuthology.orchestra.run.smithi070.stderr:adjusted tunables profile to default 2024-09-16T11:43:02.580 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T11:43:02.580 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:43:02.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T11:43:02.599 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:02.597+0000 7f05b0e53ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T11:43:02.605 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:43:02.605 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T11:43:02.623 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:02.621+0000 7f4a0522cec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T11:43:02.647 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:43:02.647 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T11:43:02.712 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T11:43:02.712 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T11:43:02.777 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:43:02.777 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-16T11:43:02.802 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:43:02.803 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-16T11:43:02.867 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:43:02.867 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-16T11:43:02.932 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-16T11:43:02.932 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-16T11:43:02.996 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 91f052e4-4c8f-4484-92ce-bd1a7f8f3efb 0 2024-09-16T11:43:03.229 INFO:teuthology.orchestra.run.smithi192.stdout:0 2024-09-16T11:43:03.240 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 568b1280-7365-4094-afe8-b0b5e766de6f 1 2024-09-16T11:43:03.412 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-09-16T11:43:03.423 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 328adfa3-c1b4-4745-9759-7327c8cbd499 2 2024-09-16T11:43:03.555 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:03.553+0000 7f4a0522cec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T11:43:03.556 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:03.553+0000 7f05b0e53ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T11:43:03.600 INFO:teuthology.orchestra.run.smithi192.stdout:2 2024-09-16T11:43:03.606 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:03.604+0000 7f4a0522cec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T11:43:03.609 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:03.606+0000 7f05b0e53ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T11:43:03.611 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 6df66ae7-e835-4d8f-bd27-243d7a9705b5 3 2024-09-16T11:43:03.790 INFO:teuthology.orchestra.run.smithi192.stdout:3 2024-09-16T11:43:03.801 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 29f52d62-d4c3-49f9-992f-23817f5622d2 4 2024-09-16T11:43:03.972 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:03.970+0000 7f4a0522cec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T11:43:03.973 INFO:teuthology.orchestra.run.smithi192.stdout:4 2024-09-16T11:43:03.984 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 0c1a275a-3c75-4e0b-bd35-d34b64d99c0a 5 2024-09-16T11:43:03.995 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:03.992+0000 7f05b0e53ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T11:43:04.053 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:04.050+0000 7f4a0522cec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T11:43:04.081 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:04.078+0000 7f05b0e53ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T11:43:04.141 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:04.138+0000 7f4a0522cec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T11:43:04.156 INFO:teuthology.orchestra.run.smithi192.stdout:5 2024-09-16T11:43:04.167 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new d79f0070-ce58-48e7-a56a-2762ec76a271 6 2024-09-16T11:43:04.173 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:04.171+0000 7f05b0e53ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T11:43:04.186 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:04.184+0000 7f4a0522cec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T11:43:04.221 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:04.218+0000 7f05b0e53ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T11:43:04.279 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:04.277+0000 7f4a0522cec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T11:43:04.319 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:04.316+0000 7f05b0e53ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T11:43:04.331 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:04.329+0000 7f4a0522cec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T11:43:04.344 INFO:teuthology.orchestra.run.smithi192.stdout:6 2024-09-16T11:43:04.358 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 537fa5df-3d0a-4dd4-8155-20ee94e41c82 7 2024-09-16T11:43:04.374 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:04.371+0000 7f05b0e53ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T11:43:04.375 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:04.373+0000 7f4a0522cec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T11:43:04.419 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:04.417+0000 7f05b0e53ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T11:43:04.468 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:04.466+0000 7f4a0522cec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T11:43:04.517 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:04.515+0000 7f4a0522cec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T11:43:04.517 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:04.514+0000 7f05b0e53ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T11:43:04.530 INFO:teuthology.orchestra.run.smithi192.stdout:7 2024-09-16T11:43:04.542 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T11:43:04.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T11:43:04.545 INFO:tasks.ceph.osd.0:Started 2024-09-16T11:43:04.545 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T11:43:04.545 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T11:43:04.548 INFO:tasks.ceph.osd.1:Started 2024-09-16T11:43:04.548 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T11:43:04.548 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T11:43:04.551 INFO:tasks.ceph.osd.2:Started 2024-09-16T11:43:04.551 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T11:43:04.551 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T11:43:04.554 INFO:tasks.ceph.osd.3:Started 2024-09-16T11:43:04.554 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T11:43:04.554 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T11:43:04.569 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:04.567+0000 7f05b0e53ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T11:43:04.571 INFO:tasks.ceph.osd.4:Started 2024-09-16T11:43:04.571 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T11:43:04.571 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T11:43:04.574 INFO:tasks.ceph.osd.5:Started 2024-09-16T11:43:04.574 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T11:43:04.574 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T11:43:04.577 INFO:tasks.ceph.osd.6:Started 2024-09-16T11:43:04.577 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-16T11:43:04.577 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-16T11:43:04.578 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:04.574+0000 7f4a0522cec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T11:43:04.580 INFO:tasks.ceph.osd.7:Started 2024-09-16T11:43:04.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:43:04.629 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:04.627+0000 7f05b0e53ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T11:43:04.660 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:04.658+0000 7f4a0522cec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T11:43:04.707 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:04.705+0000 7f4a0522cec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T11:43:04.708 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:04.706+0000 7f05b0e53ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T11:43:04.752 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:04.749+0000 7f4a0522cec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T11:43:04.752 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:04.748+0000 7f05b0e53ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T11:43:04.761 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:04.762 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":10,"fsid":"3420bcad-625f-4e15-aaff-fb4236322168","created":"2024-09-16T11:43:01.654626+0000","modified":"2024-09-16T11:43:04.526236+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T11:43:04.526236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"91f052e4-4c8f-4484-92ce-bd1a7f8f3efb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"568b1280-7365-4094-afe8-b0b5e766de6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"328adfa3-c1b4-4745-9759-7327c8cbd499","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6df66ae7-e835-4d8f-bd27-243d7a9705b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"29f52d62-d4c3-49f9-992f-23817f5622d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0c1a275a-3c75-4e0b-bd35-d34b64d99c0a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d79f0070-ce58-48e7-a56a-2762ec76a271","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"537fa5df-3d0a-4dd4-8155-20ee94e41c82","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:43:04.773 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T11:43:04.773 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T11:43:04.793 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:04.790+0000 7f05b0e53ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T11:43:05.142 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:05.139+0000 7fe04a027440 -1 Falling back to public interface 2024-09-16T11:43:05.158 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:05.155+0000 7f5fb8c4d440 -1 Falling back to public interface 2024-09-16T11:43:05.162 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:05.159+0000 7f216b06f440 -1 Falling back to public interface 2024-09-16T11:43:05.165 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:05.163+0000 7fcd30c89440 -1 Falling back to public interface 2024-09-16T11:43:05.179 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:05.176+0000 7fda86d66440 -1 Falling back to public interface 2024-09-16T11:43:05.179 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:05.176+0000 7f0f58c47440 -1 Falling back to public interface 2024-09-16T11:43:05.179 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:05.176+0000 7fda84ffd640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2024-09-16T11:43:05.188 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:05.186+0000 7fd343d2d440 -1 Falling back to public interface 2024-09-16T11:43:05.194 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:05.191+0000 7f822448d440 -1 Falling back to public interface 2024-09-16T11:43:05.366 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:05.364+0000 7f4a0522cec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T11:43:05.412 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-16T11:43:05.410+0000 7f05b0e53ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T11:43:05.579 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T11:43:05.731 INFO:teuthology.misc.health.smithi070.stdout: 2024-09-16T11:43:05.731 INFO:teuthology.misc.health.smithi070.stdout:{"epoch":10,"fsid":"3420bcad-625f-4e15-aaff-fb4236322168","created":"2024-09-16T11:43:01.654626+0000","modified":"2024-09-16T11:43:04.526236+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T11:43:04.526236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"91f052e4-4c8f-4484-92ce-bd1a7f8f3efb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"568b1280-7365-4094-afe8-b0b5e766de6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"328adfa3-c1b4-4745-9759-7327c8cbd499","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6df66ae7-e835-4d8f-bd27-243d7a9705b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"29f52d62-d4c3-49f9-992f-23817f5622d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0c1a275a-3c75-4e0b-bd35-d34b64d99c0a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d79f0070-ce58-48e7-a56a-2762ec76a271","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"537fa5df-3d0a-4dd4-8155-20ee94e41c82","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:43:05.740 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-16T11:43:06.379 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:06.377+0000 7f49e9ea8640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T11:43:06.379 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-16T11:43:06.377+0000 7f49e9ea8640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T11:43:08.582 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:08.579+0000 7fe04a027440 -1 osd.0 0 log_to_monitors true 2024-09-16T11:43:08.604 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:08.602+0000 7fda86d66440 -1 osd.4 0 log_to_monitors true 2024-09-16T11:43:08.638 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:08.636+0000 7f0f58c47440 -1 osd.6 0 log_to_monitors true 2024-09-16T11:43:08.642 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:08.640+0000 7f5fb8c4d440 -1 osd.2 0 log_to_monitors true 2024-09-16T11:43:08.668 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:08.666+0000 7f216b06f440 -1 osd.1 0 log_to_monitors true 2024-09-16T11:43:08.699 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:08.696+0000 7fd343d2d440 -1 osd.5 0 log_to_monitors true 2024-09-16T11:43:08.699 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:08.697+0000 7fcd30c89440 -1 osd.3 0 log_to_monitors true 2024-09-16T11:43:08.702 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:08.700+0000 7f822448d440 -1 osd.7 0 log_to_monitors true 2024-09-16T11:43:10.546 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:10.544+0000 7f5fb33f6640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T11:43:10.546 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:10.544+0000 7fe0447f6640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T11:43:10.550 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:10.548+0000 7f5fac3e8640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:43:10.551 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:10.549+0000 7fe03d7e8640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:43:10.557 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:10.555+0000 7f0f533f6640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T11:43:10.562 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:10.560+0000 7f0f4c3e8640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:43:10.576 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:10.574+0000 7fda817f6640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T11:43:10.580 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:10.578+0000 7fda7a7e8640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:43:11.513 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:11.510+0000 7fcd2bdf6640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T11:43:11.517 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:11.515+0000 7fcd24c84640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:43:11.522 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:11.520+0000 7f21679f9640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T11:43:11.526 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:11.524+0000 7f215f084640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:43:11.528 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:11.526+0000 7f821f5f6640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T11:43:11.533 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:11.530+0000 7f82185e8640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:43:11.540 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:11.538+0000 7fd33e7f6640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T11:43:11.546 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:11.544+0000 7fd3377d7640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:43:12.548 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T11:43:12.826 INFO:teuthology.misc.health.smithi070.stdout: 2024-09-16T11:43:12.827 INFO:teuthology.misc.health.smithi070.stdout:{"epoch":15,"fsid":"3420bcad-625f-4e15-aaff-fb4236322168","created":"2024-09-16T11:43:01.654626+0000","modified":"2024-09-16T11:43:12.692076+0000","last_up_change":"2024-09-16T11:43:11.689589+0000","last_in_change":"2024-09-16T11:43:04.526236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:43:11.406582+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"91f052e4-4c8f-4484-92ce-bd1a7f8f3efb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":33355},{"type":"v1","addr":"172.21.15.70:6801","nonce":33355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":33355},{"type":"v1","addr":"172.21.15.70:6803","nonce":33355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":33355},{"type":"v1","addr":"172.21.15.70:6807","nonce":33355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":33355},{"type":"v1","addr":"172.21.15.70:6805","nonce":33355}]},"public_addr":"172.21.15.70:6801/33355","cluster_addr":"172.21.15.70:6803/33355","heartbeat_back_addr":"172.21.15.70:6807/33355","heartbeat_front_addr":"172.21.15.70:6805/33355","state":["exists","up"]},{"osd":1,"uuid":"568b1280-7365-4094-afe8-b0b5e766de6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":33359},{"type":"v1","addr":"172.21.15.70:6817","nonce":33359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":33359},{"type":"v1","addr":"172.21.15.70:6819","nonce":33359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":33359},{"type":"v1","addr":"172.21.15.70:6823","nonce":33359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":33359},{"type":"v1","addr":"172.21.15.70:6821","nonce":33359}]},"public_addr":"172.21.15.70:6817/33359","cluster_addr":"172.21.15.70:6819/33359","heartbeat_back_addr":"172.21.15.70:6823/33359","heartbeat_front_addr":"172.21.15.70:6821/33359","state":["exists","up"]},{"osd":2,"uuid":"328adfa3-c1b4-4745-9759-7327c8cbd499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":33356},{"type":"v1","addr":"172.21.15.70:6809","nonce":33356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":33356},{"type":"v1","addr":"172.21.15.70:6811","nonce":33356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":33356},{"type":"v1","addr":"172.21.15.70:6815","nonce":33356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":33356},{"type":"v1","addr":"172.21.15.70:6813","nonce":33356}]},"public_addr":"172.21.15.70:6809/33356","cluster_addr":"172.21.15.70:6811/33356","heartbeat_back_addr":"172.21.15.70:6815/33356","heartbeat_front_addr":"172.21.15.70:6813/33356","state":["exists","up"]},{"osd":3,"uuid":"6df66ae7-e835-4d8f-bd27-243d7a9705b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":33384},{"type":"v1","addr":"172.21.15.70:6825","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":33384},{"type":"v1","addr":"172.21.15.70:6827","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":33384},{"type":"v1","addr":"172.21.15.70:6831","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":33384},{"type":"v1","addr":"172.21.15.70:6829","nonce":33384}]},"public_addr":"172.21.15.70:6825/33384","cluster_addr":"172.21.15.70:6827/33384","heartbeat_back_addr":"172.21.15.70:6831/33384","heartbeat_front_addr":"172.21.15.70:6829/33384","state":["exists","up"]},{"osd":4,"uuid":"29f52d62-d4c3-49f9-992f-23817f5622d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":33331},{"type":"v1","addr":"172.21.15.192:6803","nonce":33331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":33331},{"type":"v1","addr":"172.21.15.192:6807","nonce":33331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6813","nonce":33331},{"type":"v1","addr":"172.21.15.192:6815","nonce":33331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":33331},{"type":"v1","addr":"172.21.15.192:6811","nonce":33331}]},"public_addr":"172.21.15.192:6803/33331","cluster_addr":"172.21.15.192:6807/33331","heartbeat_back_addr":"172.21.15.192:6815/33331","heartbeat_front_addr":"172.21.15.192:6811/33331","state":["exists","up"]},{"osd":5,"uuid":"0c1a275a-3c75-4e0b-bd35-d34b64d99c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":33333},{"type":"v1","addr":"172.21.15.192:6817","nonce":33333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":33333},{"type":"v1","addr":"172.21.15.192:6819","nonce":33333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":33333},{"type":"v1","addr":"172.21.15.192:6823","nonce":33333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":33333},{"type":"v1","addr":"172.21.15.192:6821","nonce":33333}]},"public_addr":"172.21.15.192:6817/33333","cluster_addr":"172.21.15.192:6819/33333","heartbeat_back_addr":"172.21.15.192:6823/33333","heartbeat_front_addr":"172.21.15.192:6821/33333","state":["exists","up"]},{"osd":6,"uuid":"d79f0070-ce58-48e7-a56a-2762ec76a271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6801","nonce":33332},{"type":"v1","addr":"172.21.15.192:6802","nonce":33332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":33332},{"type":"v1","addr":"172.21.15.192:6806","nonce":33332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":33332},{"type":"v1","addr":"172.21.15.192:6814","nonce":33332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":33332},{"type":"v1","addr":"172.21.15.192:6810","nonce":33332}]},"public_addr":"172.21.15.192:6802/33332","cluster_addr":"172.21.15.192:6806/33332","heartbeat_back_addr":"172.21.15.192:6814/33332","heartbeat_front_addr":"172.21.15.192:6810/33332","state":["exists","up"]},{"osd":7,"uuid":"537fa5df-3d0a-4dd4-8155-20ee94e41c82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":33362},{"type":"v1","addr":"172.21.15.192:6825","nonce":33362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":33362},{"type":"v1","addr":"172.21.15.192:6827","nonce":33362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":33362},{"type":"v1","addr":"172.21.15.192:6831","nonce":33362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":33362},{"type":"v1","addr":"172.21.15.192:6829","nonce":33362}]},"public_addr":"172.21.15.192:6825/33362","cluster_addr":"172.21.15.192:6827/33362","heartbeat_back_addr":"172.21.15.192:6831/33362","heartbeat_front_addr":"172.21.15.192:6829/33362","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.581514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.674271+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.545402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.733565+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.623479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.730066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.556650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.702368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:43:12.837 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-16T11:43:12.837 INFO:tasks.ceph:Creating RBD pool 2024-09-16T11:43:12.838 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-16T11:43:13.723 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'rbd' created 2024-09-16T11:43:13.736 DEBUG:teuthology.orchestra.run.smithi070:> rbd --cluster ceph pool init rbd 2024-09-16T11:43:16.746 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T11:43:16.747 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T11:43:16.747 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T11:43:16.747 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T11:43:16.748 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T11:43:17.060 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:17.071 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":33133},{"type":"v1","addr":"172.21.15.70:6833","nonce":33133}]},"active_addr":"172.21.15.70:6833/33133","active_change":"2024-09-16T11:43:05.366479+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3618990497}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":30286296}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":172153865}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":2444281573}]}]} 2024-09-16T11:43:17.075 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T11:43:17.075 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T11:43:17.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:43:17.335 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:17.335 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":19,"fsid":"3420bcad-625f-4e15-aaff-fb4236322168","created":"2024-09-16T11:43:01.654626+0000","modified":"2024-09-16T11:43:16.729249+0000","last_up_change":"2024-09-16T11:43:11.689589+0000","last_in_change":"2024-09-16T11:43:04.526236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:43:11.406582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:43:13.125709+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"91f052e4-4c8f-4484-92ce-bd1a7f8f3efb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":33355},{"type":"v1","addr":"172.21.15.70:6801","nonce":33355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":33355},{"type":"v1","addr":"172.21.15.70:6803","nonce":33355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":33355},{"type":"v1","addr":"172.21.15.70:6807","nonce":33355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":33355},{"type":"v1","addr":"172.21.15.70:6805","nonce":33355}]},"public_addr":"172.21.15.70:6801/33355","cluster_addr":"172.21.15.70:6803/33355","heartbeat_back_addr":"172.21.15.70:6807/33355","heartbeat_front_addr":"172.21.15.70:6805/33355","state":["exists","up"]},{"osd":1,"uuid":"568b1280-7365-4094-afe8-b0b5e766de6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":33359},{"type":"v1","addr":"172.21.15.70:6817","nonce":33359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":33359},{"type":"v1","addr":"172.21.15.70:6819","nonce":33359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":33359},{"type":"v1","addr":"172.21.15.70:6823","nonce":33359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":33359},{"type":"v1","addr":"172.21.15.70:6821","nonce":33359}]},"public_addr":"172.21.15.70:6817/33359","cluster_addr":"172.21.15.70:6819/33359","heartbeat_back_addr":"172.21.15.70:6823/33359","heartbeat_front_addr":"172.21.15.70:6821/33359","state":["exists","up"]},{"osd":2,"uuid":"328adfa3-c1b4-4745-9759-7327c8cbd499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":33356},{"type":"v1","addr":"172.21.15.70:6809","nonce":33356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":33356},{"type":"v1","addr":"172.21.15.70:6811","nonce":33356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":33356},{"type":"v1","addr":"172.21.15.70:6815","nonce":33356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":33356},{"type":"v1","addr":"172.21.15.70:6813","nonce":33356}]},"public_addr":"172.21.15.70:6809/33356","cluster_addr":"172.21.15.70:6811/33356","heartbeat_back_addr":"172.21.15.70:6815/33356","heartbeat_front_addr":"172.21.15.70:6813/33356","state":["exists","up"]},{"osd":3,"uuid":"6df66ae7-e835-4d8f-bd27-243d7a9705b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":33384},{"type":"v1","addr":"172.21.15.70:6825","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":33384},{"type":"v1","addr":"172.21.15.70:6827","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":33384},{"type":"v1","addr":"172.21.15.70:6831","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":33384},{"type":"v1","addr":"172.21.15.70:6829","nonce":33384}]},"public_addr":"172.21.15.70:6825/33384","cluster_addr":"172.21.15.70:6827/33384","heartbeat_back_addr":"172.21.15.70:6831/33384","heartbeat_front_addr":"172.21.15.70:6829/33384","state":["exists","up"]},{"osd":4,"uuid":"29f52d62-d4c3-49f9-992f-23817f5622d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":33331},{"type":"v1","addr":"172.21.15.192:6803","nonce":33331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":33331},{"type":"v1","addr":"172.21.15.192:6807","nonce":33331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6813","nonce":33331},{"type":"v1","addr":"172.21.15.192:6815","nonce":33331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":33331},{"type":"v1","addr":"172.21.15.192:6811","nonce":33331}]},"public_addr":"172.21.15.192:6803/33331","cluster_addr":"172.21.15.192:6807/33331","heartbeat_back_addr":"172.21.15.192:6815/33331","heartbeat_front_addr":"172.21.15.192:6811/33331","state":["exists","up"]},{"osd":5,"uuid":"0c1a275a-3c75-4e0b-bd35-d34b64d99c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":33333},{"type":"v1","addr":"172.21.15.192:6817","nonce":33333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":33333},{"type":"v1","addr":"172.21.15.192:6819","nonce":33333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":33333},{"type":"v1","addr":"172.21.15.192:6823","nonce":33333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":33333},{"type":"v1","addr":"172.21.15.192:6821","nonce":33333}]},"public_addr":"172.21.15.192:6817/33333","cluster_addr":"172.21.15.192:6819/33333","heartbeat_back_addr":"172.21.15.192:6823/33333","heartbeat_front_addr":"172.21.15.192:6821/33333","state":["exists","up"]},{"osd":6,"uuid":"d79f0070-ce58-48e7-a56a-2762ec76a271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6801","nonce":33332},{"type":"v1","addr":"172.21.15.192:6802","nonce":33332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":33332},{"type":"v1","addr":"172.21.15.192:6806","nonce":33332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":33332},{"type":"v1","addr":"172.21.15.192:6814","nonce":33332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":33332},{"type":"v1","addr":"172.21.15.192:6810","nonce":33332}]},"public_addr":"172.21.15.192:6802/33332","cluster_addr":"172.21.15.192:6806/33332","heartbeat_back_addr":"172.21.15.192:6814/33332","heartbeat_front_addr":"172.21.15.192:6810/33332","state":["exists","up"]},{"osd":7,"uuid":"537fa5df-3d0a-4dd4-8155-20ee94e41c82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":33362},{"type":"v1","addr":"172.21.15.192:6825","nonce":33362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":33362},{"type":"v1","addr":"172.21.15.192:6827","nonce":33362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":33362},{"type":"v1","addr":"172.21.15.192:6831","nonce":33362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":33362},{"type":"v1","addr":"172.21.15.192:6829","nonce":33362}]},"public_addr":"172.21.15.192:6825/33362","cluster_addr":"172.21.15.192:6827/33362","heartbeat_back_addr":"172.21.15.192:6831/33362","heartbeat_front_addr":"172.21.15.192:6829/33362","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.581514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.674271+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.545402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.733565+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.623479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.730066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.556650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.702368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:43:17.347 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T11:43:17.347 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:43:17.607 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:17.608 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":19,"fsid":"3420bcad-625f-4e15-aaff-fb4236322168","created":"2024-09-16T11:43:01.654626+0000","modified":"2024-09-16T11:43:16.729249+0000","last_up_change":"2024-09-16T11:43:11.689589+0000","last_in_change":"2024-09-16T11:43:04.526236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:43:11.406582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:43:13.125709+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"91f052e4-4c8f-4484-92ce-bd1a7f8f3efb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":33355},{"type":"v1","addr":"172.21.15.70:6801","nonce":33355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":33355},{"type":"v1","addr":"172.21.15.70:6803","nonce":33355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":33355},{"type":"v1","addr":"172.21.15.70:6807","nonce":33355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":33355},{"type":"v1","addr":"172.21.15.70:6805","nonce":33355}]},"public_addr":"172.21.15.70:6801/33355","cluster_addr":"172.21.15.70:6803/33355","heartbeat_back_addr":"172.21.15.70:6807/33355","heartbeat_front_addr":"172.21.15.70:6805/33355","state":["exists","up"]},{"osd":1,"uuid":"568b1280-7365-4094-afe8-b0b5e766de6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":33359},{"type":"v1","addr":"172.21.15.70:6817","nonce":33359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":33359},{"type":"v1","addr":"172.21.15.70:6819","nonce":33359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":33359},{"type":"v1","addr":"172.21.15.70:6823","nonce":33359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":33359},{"type":"v1","addr":"172.21.15.70:6821","nonce":33359}]},"public_addr":"172.21.15.70:6817/33359","cluster_addr":"172.21.15.70:6819/33359","heartbeat_back_addr":"172.21.15.70:6823/33359","heartbeat_front_addr":"172.21.15.70:6821/33359","state":["exists","up"]},{"osd":2,"uuid":"328adfa3-c1b4-4745-9759-7327c8cbd499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":33356},{"type":"v1","addr":"172.21.15.70:6809","nonce":33356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":33356},{"type":"v1","addr":"172.21.15.70:6811","nonce":33356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":33356},{"type":"v1","addr":"172.21.15.70:6815","nonce":33356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":33356},{"type":"v1","addr":"172.21.15.70:6813","nonce":33356}]},"public_addr":"172.21.15.70:6809/33356","cluster_addr":"172.21.15.70:6811/33356","heartbeat_back_addr":"172.21.15.70:6815/33356","heartbeat_front_addr":"172.21.15.70:6813/33356","state":["exists","up"]},{"osd":3,"uuid":"6df66ae7-e835-4d8f-bd27-243d7a9705b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":33384},{"type":"v1","addr":"172.21.15.70:6825","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":33384},{"type":"v1","addr":"172.21.15.70:6827","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":33384},{"type":"v1","addr":"172.21.15.70:6831","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":33384},{"type":"v1","addr":"172.21.15.70:6829","nonce":33384}]},"public_addr":"172.21.15.70:6825/33384","cluster_addr":"172.21.15.70:6827/33384","heartbeat_back_addr":"172.21.15.70:6831/33384","heartbeat_front_addr":"172.21.15.70:6829/33384","state":["exists","up"]},{"osd":4,"uuid":"29f52d62-d4c3-49f9-992f-23817f5622d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":33331},{"type":"v1","addr":"172.21.15.192:6803","nonce":33331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":33331},{"type":"v1","addr":"172.21.15.192:6807","nonce":33331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6813","nonce":33331},{"type":"v1","addr":"172.21.15.192:6815","nonce":33331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":33331},{"type":"v1","addr":"172.21.15.192:6811","nonce":33331}]},"public_addr":"172.21.15.192:6803/33331","cluster_addr":"172.21.15.192:6807/33331","heartbeat_back_addr":"172.21.15.192:6815/33331","heartbeat_front_addr":"172.21.15.192:6811/33331","state":["exists","up"]},{"osd":5,"uuid":"0c1a275a-3c75-4e0b-bd35-d34b64d99c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":33333},{"type":"v1","addr":"172.21.15.192:6817","nonce":33333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":33333},{"type":"v1","addr":"172.21.15.192:6819","nonce":33333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":33333},{"type":"v1","addr":"172.21.15.192:6823","nonce":33333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":33333},{"type":"v1","addr":"172.21.15.192:6821","nonce":33333}]},"public_addr":"172.21.15.192:6817/33333","cluster_addr":"172.21.15.192:6819/33333","heartbeat_back_addr":"172.21.15.192:6823/33333","heartbeat_front_addr":"172.21.15.192:6821/33333","state":["exists","up"]},{"osd":6,"uuid":"d79f0070-ce58-48e7-a56a-2762ec76a271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6801","nonce":33332},{"type":"v1","addr":"172.21.15.192:6802","nonce":33332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":33332},{"type":"v1","addr":"172.21.15.192:6806","nonce":33332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":33332},{"type":"v1","addr":"172.21.15.192:6814","nonce":33332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":33332},{"type":"v1","addr":"172.21.15.192:6810","nonce":33332}]},"public_addr":"172.21.15.192:6802/33332","cluster_addr":"172.21.15.192:6806/33332","heartbeat_back_addr":"172.21.15.192:6814/33332","heartbeat_front_addr":"172.21.15.192:6810/33332","state":["exists","up"]},{"osd":7,"uuid":"537fa5df-3d0a-4dd4-8155-20ee94e41c82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":33362},{"type":"v1","addr":"172.21.15.192:6825","nonce":33362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":33362},{"type":"v1","addr":"172.21.15.192:6827","nonce":33362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":33362},{"type":"v1","addr":"172.21.15.192:6831","nonce":33362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":33362},{"type":"v1","addr":"172.21.15.192:6829","nonce":33362}]},"public_addr":"172.21.15.192:6825/33362","cluster_addr":"172.21.15.192:6827/33362","heartbeat_back_addr":"172.21.15.192:6831/33362","heartbeat_front_addr":"172.21.15.192:6829/33362","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.581514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.674271+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.545402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.733565+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.623479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.730066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.556650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.702368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:43:17.620 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T11:43:17.621 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T11:43:17.621 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T11:43:17.621 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T11:43:17.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T11:43:17.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T11:43:17.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T11:43:17.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T11:43:17.850 INFO:teuthology.orchestra.run.smithi070.stdout:55834574851 2024-09-16T11:43:17.850 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T11:43:17.856 INFO:teuthology.orchestra.run.smithi070.stdout:55834574851 2024-09-16T11:43:17.856 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T11:43:17.857 INFO:teuthology.orchestra.run.smithi070.stdout:60129542149 2024-09-16T11:43:17.857 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T11:43:17.858 INFO:teuthology.orchestra.run.smithi070.stdout:55834574851 2024-09-16T11:43:17.858 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T11:43:17.859 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-16T11:43:17.860 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T11:43:17.861 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-16T11:43:17.861 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T11:43:17.862 INFO:teuthology.orchestra.run.smithi070.stdout:60129542147 2024-09-16T11:43:17.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T11:43:17.863 INFO:teuthology.orchestra.run.smithi070.stdout:55834574851 2024-09-16T11:43:17.863 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T11:43:18.367 INFO:teuthology.orchestra.run.smithi070.stdout:55834574850 2024-09-16T11:43:18.372 INFO:teuthology.orchestra.run.smithi070.stdout:55834574850 2024-09-16T11:43:18.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2024-09-16T11:43:18.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.2 2024-09-16T11:43:18.415 INFO:teuthology.orchestra.run.smithi070.stdout:60129542146 2024-09-16T11:43:18.420 INFO:teuthology.orchestra.run.smithi070.stdout:60129542147 2024-09-16T11:43:18.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542146 for osd.7 2024-09-16T11:43:18.433 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542147 for osd.3 2024-09-16T11:43:18.450 INFO:teuthology.orchestra.run.smithi070.stdout:60129542145 2024-09-16T11:43:18.451 INFO:teuthology.orchestra.run.smithi070.stdout:55834574850 2024-09-16T11:43:18.453 INFO:teuthology.orchestra.run.smithi070.stdout:60129542146 2024-09-16T11:43:18.461 INFO:teuthology.orchestra.run.smithi070.stdout:55834574850 2024-09-16T11:43:18.466 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542145 for osd.5 2024-09-16T11:43:18.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2024-09-16T11:43:18.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542146 for osd.1 2024-09-16T11:43:18.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2024-09-16T11:43:19.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T11:43:19.393 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T11:43:19.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T11:43:19.435 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T11:43:19.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T11:43:19.467 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T11:43:19.472 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T11:43:19.477 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T11:43:19.868 INFO:teuthology.orchestra.run.smithi070.stdout:55834574851 2024-09-16T11:43:19.890 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2024-09-16T11:43:19.890 DEBUG:teuthology.parallel:result is None 2024-09-16T11:43:19.941 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-16T11:43:19.946 INFO:teuthology.orchestra.run.smithi070.stdout:55834574851 2024-09-16T11:43:19.950 INFO:teuthology.orchestra.run.smithi070.stdout:60129542148 2024-09-16T11:43:19.956 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.1 2024-09-16T11:43:19.956 DEBUG:teuthology.parallel:result is None 2024-09-16T11:43:19.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2024-09-16T11:43:19.966 DEBUG:teuthology.parallel:result is None 2024-09-16T11:43:19.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.7 2024-09-16T11:43:19.969 DEBUG:teuthology.parallel:result is None 2024-09-16T11:43:19.994 INFO:teuthology.orchestra.run.smithi070.stdout:60129542149 2024-09-16T11:43:20.005 INFO:teuthology.orchestra.run.smithi070.stdout:55834574851 2024-09-16T11:43:20.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2024-09-16T11:43:20.007 DEBUG:teuthology.parallel:result is None 2024-09-16T11:43:20.008 INFO:teuthology.orchestra.run.smithi070.stdout:55834574851 2024-09-16T11:43:20.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2024-09-16T11:43:20.021 INFO:teuthology.orchestra.run.smithi070.stdout:60129542147 2024-09-16T11:43:20.021 DEBUG:teuthology.parallel:result is None 2024-09-16T11:43:20.024 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2024-09-16T11:43:20.024 DEBUG:teuthology.parallel:result is None 2024-09-16T11:43:20.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.5 2024-09-16T11:43:20.032 DEBUG:teuthology.parallel:result is None 2024-09-16T11:43:20.032 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T11:43:20.032 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:43:20.329 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:20.330 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:43:20.341 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-16T11:43:19.373171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2397100,"kb_used_data":1588,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752577620,"statfs":{"total":773094113280,"available":770639482880,"internally_reserved":0,"allocated":1626112,"data_stored":1684570,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.732844+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:43:16.732844+0000","last_peered":"2024-09-16T11:43:16.732844+0000","last_clean":"2024-09-16T11:43:16.732844+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:43:16.732844+0000","last_undegraded":"2024-09-16T11:43:16.732844+0000","last_fullsized":"2024-09-16T11:43:16.732844+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.740771+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:43:16.740771+0000","last_peered":"2024-09-16T11:43:16.740771+0000","last_clean":"2024-09-16T11:43:16.740771+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:43:16.740771+0000","last_undegraded":"2024-09-16T11:43:16.740771+0000","last_fullsized":"2024-09-16T11:43:16.740771+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.738844+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:43:16.738844+0000","last_peered":"2024-09-16T11:43:16.738844+0000","last_clean":"2024-09-16T11:43:16.738844+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:43:16.738844+0000","last_undegraded":"2024-09-16T11:43:16.738844+0000","last_fullsized":"2024-09-16T11:43:16.738844+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.740722+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:43:16.740722+0000","last_peered":"2024-09-16T11:43:16.740722+0000","last_clean":"2024-09-16T11:43:16.740722+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:43:16.740722+0000","last_undegraded":"2024-09-16T11:43:16.740722+0000","last_fullsized":"2024-09-16T11:43:16.740722+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.739428+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:43:16.739428+0000","last_peered":"2024-09-16T11:43:16.739428+0000","last_clean":"2024-09-16T11:43:16.739428+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:43:16.739428+0000","last_undegraded":"2024-09-16T11:43:16.739428+0000","last_fullsized":"2024-09-16T11:43:16.739428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.732633+0000","last_change":"2024-09-16T11:43:16.732834+0000","last_active":"2024-09-16T11:43:16.732633+0000","last_peered":"2024-09-16T11:43:16.732633+0000","last_clean":"2024-09-16T11:43:16.732633+0000","last_became_active":"2024-09-16T11:43:14.724728+0000","last_became_peered":"2024-09-16T11:43:14.724728+0000","last_unstale":"2024-09-16T11:43:16.732633+0000","last_undegraded":"2024-09-16T11:43:16.732633+0000","last_fullsized":"2024-09-16T11:43:16.732633+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:47.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.738792+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:43:16.738792+0000","last_peered":"2024-09-16T11:43:16.738792+0000","last_clean":"2024-09-16T11:43:16.738792+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:43:16.738792+0000","last_undegraded":"2024-09-16T11:43:16.738792+0000","last_fullsized":"2024-09-16T11:43:16.738792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.738495+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:43:16.738495+0000","last_peered":"2024-09-16T11:43:16.738495+0000","last_clean":"2024-09-16T11:43:16.738495+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:43:16.738495+0000","last_undegraded":"2024-09-16T11:43:16.738495+0000","last_fullsized":"2024-09-16T11:43:16.738495+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'72","reported_seq":102,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.271005+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:43:16.271005+0000","last_peered":"2024-09-16T11:43:16.271005+0000","last_clean":"2024-09-16T11:43:16.271005+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:43:16.271005+0000","last_undegraded":"2024-09-16T11:43:16.271005+0000","last_fullsized":"2024-09-16T11:43:16.271005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":245760,"data_stored":653461,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297552,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074288,"statfs":{"total":96636764160,"available":96332070912,"internally_reserved":0,"allocated":192512,"data_stored":63112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301708,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070132,"statfs":{"total":96636764160,"available":96327815168,"internally_reserved":0,"allocated":188416,"data_stored":63093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.191}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301712,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070128,"statfs":{"total":96636764160,"available":96327811072,"internally_reserved":0,"allocated":192512,"data_stored":63112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":184320,"data_stored":62145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297540,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":188416,"data_stored":63093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301700,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070140,"statfs":{"total":96636764160,"available":96327823360,"internally_reserved":0,"allocated":188416,"data_stored":63093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301756,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070084,"statfs":{"total":96636764160,"available":96327766016,"internally_reserved":0,"allocated":245760,"data_stored":653461,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:43:20.342 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:43:20.600 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:20.601 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:43:20.612 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-16T11:43:19.373171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2397100,"kb_used_data":1588,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752577620,"statfs":{"total":773094113280,"available":770639482880,"internally_reserved":0,"allocated":1626112,"data_stored":1684570,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.732844+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:43:16.732844+0000","last_peered":"2024-09-16T11:43:16.732844+0000","last_clean":"2024-09-16T11:43:16.732844+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:43:16.732844+0000","last_undegraded":"2024-09-16T11:43:16.732844+0000","last_fullsized":"2024-09-16T11:43:16.732844+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.740771+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:43:16.740771+0000","last_peered":"2024-09-16T11:43:16.740771+0000","last_clean":"2024-09-16T11:43:16.740771+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:43:16.740771+0000","last_undegraded":"2024-09-16T11:43:16.740771+0000","last_fullsized":"2024-09-16T11:43:16.740771+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.738844+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:43:16.738844+0000","last_peered":"2024-09-16T11:43:16.738844+0000","last_clean":"2024-09-16T11:43:16.738844+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:43:16.738844+0000","last_undegraded":"2024-09-16T11:43:16.738844+0000","last_fullsized":"2024-09-16T11:43:16.738844+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.740722+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:43:16.740722+0000","last_peered":"2024-09-16T11:43:16.740722+0000","last_clean":"2024-09-16T11:43:16.740722+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:43:16.740722+0000","last_undegraded":"2024-09-16T11:43:16.740722+0000","last_fullsized":"2024-09-16T11:43:16.740722+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.739428+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:43:16.739428+0000","last_peered":"2024-09-16T11:43:16.739428+0000","last_clean":"2024-09-16T11:43:16.739428+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:43:16.739428+0000","last_undegraded":"2024-09-16T11:43:16.739428+0000","last_fullsized":"2024-09-16T11:43:16.739428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.732633+0000","last_change":"2024-09-16T11:43:16.732834+0000","last_active":"2024-09-16T11:43:16.732633+0000","last_peered":"2024-09-16T11:43:16.732633+0000","last_clean":"2024-09-16T11:43:16.732633+0000","last_became_active":"2024-09-16T11:43:14.724728+0000","last_became_peered":"2024-09-16T11:43:14.724728+0000","last_unstale":"2024-09-16T11:43:16.732633+0000","last_undegraded":"2024-09-16T11:43:16.732633+0000","last_fullsized":"2024-09-16T11:43:16.732633+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:47.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.738792+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:43:16.738792+0000","last_peered":"2024-09-16T11:43:16.738792+0000","last_clean":"2024-09-16T11:43:16.738792+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:43:16.738792+0000","last_undegraded":"2024-09-16T11:43:16.738792+0000","last_fullsized":"2024-09-16T11:43:16.738792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.738495+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:43:16.738495+0000","last_peered":"2024-09-16T11:43:16.738495+0000","last_clean":"2024-09-16T11:43:16.738495+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:43:16.738495+0000","last_undegraded":"2024-09-16T11:43:16.738495+0000","last_fullsized":"2024-09-16T11:43:16.738495+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'72","reported_seq":102,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-16T11:43:16.271005+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:43:16.271005+0000","last_peered":"2024-09-16T11:43:16.271005+0000","last_clean":"2024-09-16T11:43:16.271005+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:43:16.271005+0000","last_undegraded":"2024-09-16T11:43:16.271005+0000","last_fullsized":"2024-09-16T11:43:16.271005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297596,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074244,"statfs":{"total":96636764160,"available":96332025856,"internally_reserved":0,"allocated":245760,"data_stored":653461,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297552,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074288,"statfs":{"total":96636764160,"available":96332070912,"internally_reserved":0,"allocated":192512,"data_stored":63112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301708,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070132,"statfs":{"total":96636764160,"available":96327815168,"internally_reserved":0,"allocated":188416,"data_stored":63093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.191}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301712,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070128,"statfs":{"total":96636764160,"available":96327811072,"internally_reserved":0,"allocated":192512,"data_stored":63112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":184320,"data_stored":62145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297540,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074300,"statfs":{"total":96636764160,"available":96332083200,"internally_reserved":0,"allocated":188416,"data_stored":63093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301700,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070140,"statfs":{"total":96636764160,"available":96327823360,"internally_reserved":0,"allocated":188416,"data_stored":63093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301756,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070084,"statfs":{"total":96636764160,"available":96327766016,"internally_reserved":0,"allocated":245760,"data_stored":653461,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:43:20.614 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T11:43:20.614 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T11:43:20.614 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T11:43:20.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T11:43:20.905 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:20.906 INFO:teuthology.orchestra.run.smithi070.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T11:43:20.917 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T11:43:20.918 INFO:teuthology.run_tasks:Running task exec... 2024-09-16T11:43:20.943 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T11:43:20.943 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T11:43:20.943 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create cache 4' 2024-09-16T11:43:21.436 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cache' created 2024-09-16T11:43:21.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier add rbd cache' 2024-09-16T11:43:22.437 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cache' is now (or already was) a tier of 'rbd' 2024-09-16T11:43:22.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier cache-mode cache writeback' 2024-09-16T11:43:23.441 INFO:teuthology.orchestra.run.smithi070.stderr:set cache-mode for pool 'cache' to writeback 2024-09-16T11:43:23.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier set-overlay rbd cache' 2024-09-16T11:43:24.446 INFO:teuthology.orchestra.run.smithi070.stderr:overlay for 'rbd' is now (or already was) 'cache' 2024-09-16T11:43:24.461 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_type bloom' 2024-09-16T11:43:25.453 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 3 hit_set_type to bloom 2024-09-16T11:43:25.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_count 8' 2024-09-16T11:43:26.452 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 3 hit_set_count to 8 2024-09-16T11:43:26.467 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_period 60' 2024-09-16T11:43:27.463 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 3 hit_set_period to 60 2024-09-16T11:43:27.478 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache target_max_objects 250' 2024-09-16T11:43:28.464 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 3 target_max_objects to 250 2024-09-16T11:43:28.480 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-16T11:43:28.491 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-16T11:43:28.491 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-16T11:43:28.491 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-16T11:43:28.492 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-16T11:43:28.773 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-16T11:43:28.773 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T11:43:28.774 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:43:29.031 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:29.031 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:43:29.042 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-16T11:43:28.461167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397620,"kb_used_data":2092,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752577100,"statfs":{"total":773094113280,"available":770638950400,"internally_reserved":0,"allocated":2142208,"data_stored":1947250,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.018243"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T11:43:24.043614+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:43:24.043614+0000","last_peered":"2024-09-16T11:43:24.043614+0000","last_clean":"2024-09-16T11:43:24.043614+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:43:24.043614+0000","last_undegraded":"2024-09-16T11:43:24.043614+0000","last_fullsized":"2024-09-16T11:43:24.043614+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.702971+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:43:26.702971+0000","last_peered":"2024-09-16T11:43:26.702971+0000","last_clean":"2024-09-16T11:43:26.702971+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:43:26.702971+0000","last_undegraded":"2024-09-16T11:43:26.702971+0000","last_fullsized":"2024-09-16T11:43:26.702971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.452911+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:43:26.452911+0000","last_peered":"2024-09-16T11:43:26.452911+0000","last_clean":"2024-09-16T11:43:26.452911+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:43:26.452911+0000","last_undegraded":"2024-09-16T11:43:26.452911+0000","last_fullsized":"2024-09-16T11:43:26.452911+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.702930+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:43:26.702930+0000","last_peered":"2024-09-16T11:43:26.702930+0000","last_clean":"2024-09-16T11:43:26.702930+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:43:26.702930+0000","last_undegraded":"2024-09-16T11:43:26.702930+0000","last_fullsized":"2024-09-16T11:43:26.702930+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T11:43:24.446961+0000","last_change":"2024-09-16T11:43:22.438212+0000","last_active":"2024-09-16T11:43:24.446961+0000","last_peered":"2024-09-16T11:43:24.446961+0000","last_clean":"2024-09-16T11:43:24.446961+0000","last_became_active":"2024-09-16T11:43:22.437868+0000","last_became_peered":"2024-09-16T11:43:22.437868+0000","last_unstale":"2024-09-16T11:43:24.446961+0000","last_undegraded":"2024-09-16T11:43:24.446961+0000","last_fullsized":"2024-09-16T11:43:24.446961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:35.498933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:43:25.603282+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:43:25.603282+0000","last_peered":"2024-09-16T11:43:25.603282+0000","last_clean":"2024-09-16T11:43:25.603282+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:43:25.603282+0000","last_undegraded":"2024-09-16T11:43:25.603282+0000","last_fullsized":"2024-09-16T11:43:25.603282+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.702930+0000","last_change":"2024-09-16T11:43:22.440136+0000","last_active":"2024-09-16T11:43:26.702930+0000","last_peered":"2024-09-16T11:43:26.702930+0000","last_clean":"2024-09-16T11:43:26.702930+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:43:26.702930+0000","last_undegraded":"2024-09-16T11:43:26.702930+0000","last_fullsized":"2024-09-16T11:43:26.702930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:18.461948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T11:43:24.443980+0000","last_change":"2024-09-16T11:43:16.732834+0000","last_active":"2024-09-16T11:43:24.443980+0000","last_peered":"2024-09-16T11:43:24.443980+0000","last_clean":"2024-09-16T11:43:24.443980+0000","last_became_active":"2024-09-16T11:43:14.724728+0000","last_became_peered":"2024-09-16T11:43:14.724728+0000","last_unstale":"2024-09-16T11:43:24.443980+0000","last_undegraded":"2024-09-16T11:43:24.443980+0000","last_fullsized":"2024-09-16T11:43:24.443980+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:47.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T11:43:23.439724+0000","last_change":"2024-09-16T11:43:22.437965+0000","last_active":"2024-09-16T11:43:23.439724+0000","last_peered":"2024-09-16T11:43:23.439724+0000","last_clean":"2024-09-16T11:43:23.439724+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:43:23.439724+0000","last_undegraded":"2024-09-16T11:43:23.439724+0000","last_fullsized":"2024-09-16T11:43:23.439724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:38:39.799255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.452903+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:43:26.452903+0000","last_peered":"2024-09-16T11:43:26.452903+0000","last_clean":"2024-09-16T11:43:26.452903+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:43:26.452903+0000","last_undegraded":"2024-09-16T11:43:26.452903+0000","last_fullsized":"2024-09-16T11:43:26.452903+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.450186+0000","last_change":"2024-09-16T11:43:22.587076+0000","last_active":"2024-09-16T11:43:26.450186+0000","last_peered":"2024-09-16T11:43:26.450186+0000","last_clean":"2024-09-16T11:43:26.450186+0000","last_became_active":"2024-09-16T11:43:22.586861+0000","last_became_peered":"2024-09-16T11:43:22.586861+0000","last_unstale":"2024-09-16T11:43:26.450186+0000","last_undegraded":"2024-09-16T11:43:26.450186+0000","last_fullsized":"2024-09-16T11:43:26.450186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:54:46.029558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:43:25.603192+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:43:25.603192+0000","last_peered":"2024-09-16T11:43:25.603192+0000","last_clean":"2024-09-16T11:43:25.603192+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:43:25.603192+0000","last_undegraded":"2024-09-16T11:43:25.603192+0000","last_fullsized":"2024-09-16T11:43:25.603192+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":108,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.452974+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:43:26.452974+0000","last_peered":"2024-09-16T11:43:26.452974+0000","last_clean":"2024-09-16T11:43:26.452974+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:43:26.452974+0000","last_undegraded":"2024-09-16T11:43:26.452974+0000","last_fullsized":"2024-09-16T11:43:26.452974+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":319488,"data_stored":690946,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297624,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074216,"statfs":{"total":96636764160,"available":96331997184,"internally_reserved":0,"allocated":266240,"data_stored":100597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":262144,"data_stored":100578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301784,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070056,"statfs":{"total":96636764160,"available":96327737344,"internally_reserved":0,"allocated":266240,"data_stored":100597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":80883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297608,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074232,"statfs":{"total":96636764160,"available":96332013568,"internally_reserved":0,"allocated":249856,"data_stored":94427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301768,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070072,"statfs":{"total":96636764160,"available":96327753728,"internally_reserved":0,"allocated":249856,"data_stored":94427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301820,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070020,"statfs":{"total":96636764160,"available":96327700480,"internally_reserved":0,"allocated":307200,"data_stored":684795,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:43:29.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:43:29.300 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:29.301 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:43:29.312 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-16T11:43:28.461167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397620,"kb_used_data":2092,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752577100,"statfs":{"total":773094113280,"available":770638950400,"internally_reserved":0,"allocated":2142208,"data_stored":1947250,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.018243"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T11:43:24.043614+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:43:24.043614+0000","last_peered":"2024-09-16T11:43:24.043614+0000","last_clean":"2024-09-16T11:43:24.043614+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:43:24.043614+0000","last_undegraded":"2024-09-16T11:43:24.043614+0000","last_fullsized":"2024-09-16T11:43:24.043614+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.702971+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:43:26.702971+0000","last_peered":"2024-09-16T11:43:26.702971+0000","last_clean":"2024-09-16T11:43:26.702971+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:43:26.702971+0000","last_undegraded":"2024-09-16T11:43:26.702971+0000","last_fullsized":"2024-09-16T11:43:26.702971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.452911+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:43:26.452911+0000","last_peered":"2024-09-16T11:43:26.452911+0000","last_clean":"2024-09-16T11:43:26.452911+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:43:26.452911+0000","last_undegraded":"2024-09-16T11:43:26.452911+0000","last_fullsized":"2024-09-16T11:43:26.452911+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.702930+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:43:26.702930+0000","last_peered":"2024-09-16T11:43:26.702930+0000","last_clean":"2024-09-16T11:43:26.702930+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:43:26.702930+0000","last_undegraded":"2024-09-16T11:43:26.702930+0000","last_fullsized":"2024-09-16T11:43:26.702930+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T11:43:24.446961+0000","last_change":"2024-09-16T11:43:22.438212+0000","last_active":"2024-09-16T11:43:24.446961+0000","last_peered":"2024-09-16T11:43:24.446961+0000","last_clean":"2024-09-16T11:43:24.446961+0000","last_became_active":"2024-09-16T11:43:22.437868+0000","last_became_peered":"2024-09-16T11:43:22.437868+0000","last_unstale":"2024-09-16T11:43:24.446961+0000","last_undegraded":"2024-09-16T11:43:24.446961+0000","last_fullsized":"2024-09-16T11:43:24.446961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:35.498933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:43:25.603282+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:43:25.603282+0000","last_peered":"2024-09-16T11:43:25.603282+0000","last_clean":"2024-09-16T11:43:25.603282+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:43:25.603282+0000","last_undegraded":"2024-09-16T11:43:25.603282+0000","last_fullsized":"2024-09-16T11:43:25.603282+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.702930+0000","last_change":"2024-09-16T11:43:22.440136+0000","last_active":"2024-09-16T11:43:26.702930+0000","last_peered":"2024-09-16T11:43:26.702930+0000","last_clean":"2024-09-16T11:43:26.702930+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:43:26.702930+0000","last_undegraded":"2024-09-16T11:43:26.702930+0000","last_fullsized":"2024-09-16T11:43:26.702930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:18.461948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T11:43:24.443980+0000","last_change":"2024-09-16T11:43:16.732834+0000","last_active":"2024-09-16T11:43:24.443980+0000","last_peered":"2024-09-16T11:43:24.443980+0000","last_clean":"2024-09-16T11:43:24.443980+0000","last_became_active":"2024-09-16T11:43:14.724728+0000","last_became_peered":"2024-09-16T11:43:14.724728+0000","last_unstale":"2024-09-16T11:43:24.443980+0000","last_undegraded":"2024-09-16T11:43:24.443980+0000","last_fullsized":"2024-09-16T11:43:24.443980+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:47.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T11:43:23.439724+0000","last_change":"2024-09-16T11:43:22.437965+0000","last_active":"2024-09-16T11:43:23.439724+0000","last_peered":"2024-09-16T11:43:23.439724+0000","last_clean":"2024-09-16T11:43:23.439724+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:43:23.439724+0000","last_undegraded":"2024-09-16T11:43:23.439724+0000","last_fullsized":"2024-09-16T11:43:23.439724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:38:39.799255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.452903+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:43:26.452903+0000","last_peered":"2024-09-16T11:43:26.452903+0000","last_clean":"2024-09-16T11:43:26.452903+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:43:26.452903+0000","last_undegraded":"2024-09-16T11:43:26.452903+0000","last_fullsized":"2024-09-16T11:43:26.452903+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.450186+0000","last_change":"2024-09-16T11:43:22.587076+0000","last_active":"2024-09-16T11:43:26.450186+0000","last_peered":"2024-09-16T11:43:26.450186+0000","last_clean":"2024-09-16T11:43:26.450186+0000","last_became_active":"2024-09-16T11:43:22.586861+0000","last_became_peered":"2024-09-16T11:43:22.586861+0000","last_unstale":"2024-09-16T11:43:26.450186+0000","last_undegraded":"2024-09-16T11:43:26.450186+0000","last_fullsized":"2024-09-16T11:43:26.450186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:54:46.029558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:43:25.603192+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:43:25.603192+0000","last_peered":"2024-09-16T11:43:25.603192+0000","last_clean":"2024-09-16T11:43:25.603192+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:43:25.603192+0000","last_undegraded":"2024-09-16T11:43:25.603192+0000","last_fullsized":"2024-09-16T11:43:25.603192+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":108,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:43:26.452974+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:43:26.452974+0000","last_peered":"2024-09-16T11:43:26.452974+0000","last_clean":"2024-09-16T11:43:26.452974+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:43:26.452974+0000","last_undegraded":"2024-09-16T11:43:26.452974+0000","last_fullsized":"2024-09-16T11:43:26.452974+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":319488,"data_stored":690946,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297624,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074216,"statfs":{"total":96636764160,"available":96331997184,"internally_reserved":0,"allocated":266240,"data_stored":100597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":262144,"data_stored":100578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301784,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070056,"statfs":{"total":96636764160,"available":96327737344,"internally_reserved":0,"allocated":266240,"data_stored":100597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":80883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297608,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074232,"statfs":{"total":96636764160,"available":96332013568,"internally_reserved":0,"allocated":249856,"data_stored":94427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301768,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070072,"statfs":{"total":96636764160,"available":96327753728,"internally_reserved":0,"allocated":249856,"data_stored":94427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301820,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070020,"statfs":{"total":96636764160,"available":96327700480,"internally_reserved":0,"allocated":307200,"data_stored":684795,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:43:29.313 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T11:43:29.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-16T11:43:29.573 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 27 2024-09-16T11:43:29.573 INFO:teuthology.orchestra.run.smithi070.stdout:fsid 3420bcad-625f-4e15-aaff-fb4236322168 2024-09-16T11:43:29.573 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-09-16T11:43:01.654626+0000 2024-09-16T11:43:29.573 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-09-16T11:43:28.459122+0000 2024-09-16T11:43:29.573 INFO:teuthology.orchestra.run.smithi070.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T11:43:29.573 INFO:teuthology.orchestra.run.smithi070.stdout:crush_version 5 2024-09-16T11:43:29.573 INFO:teuthology.orchestra.run.smithi070.stdout:full_ratio 0.95 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:backfillfull_ratio 0.9 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:nearfull_ratio 0.85 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:require_min_compat_client luminous 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:min_compat_client jewel 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:require_osd_release quincy 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:stretch_mode_enabled false 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 23 lfor 23/23/23 flags hashpspool,selfmanaged_snaps tiers 3 read_tier 3 write_tier 3 stripe_width 0 application rbd 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:pool 3 'cache' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 4 pgp_num 4 autoscale_mode off last_change 27 lfor 23/23/23 flags hashpspool,incomplete_clones,selfmanaged_snaps tier_of 2 cache_mode writeback target_objects 250 hit_set bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 60s x8 decay_rate 0 search_last_n 0 stripe_width 0 application rbd 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:max_osd 8 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 up in weight 1 up_from 13 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6800/33355,v1:172.21.15.70:6801/33355] [v2:172.21.15.70:6802/33355,v1:172.21.15.70:6803/33355] exists,up 91f052e4-4c8f-4484-92ce-bd1a7f8f3efb 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6816/33359,v1:172.21.15.70:6817/33359] [v2:172.21.15.70:6818/33359,v1:172.21.15.70:6819/33359] exists,up 568b1280-7365-4094-afe8-b0b5e766de6f 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6808/33356,v1:172.21.15.70:6809/33356] [v2:172.21.15.70:6810/33356,v1:172.21.15.70:6811/33356] exists,up 328adfa3-c1b4-4745-9759-7327c8cbd499 2024-09-16T11:43:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6824/33384,v1:172.21.15.70:6825/33384] [v2:172.21.15.70:6826/33384,v1:172.21.15.70:6827/33384] exists,up 6df66ae7-e835-4d8f-bd27-243d7a9705b5 2024-09-16T11:43:29.575 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 up in weight 1 up_from 13 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/33331,v1:172.21.15.192:6803/33331] [v2:172.21.15.192:6805/33331,v1:172.21.15.192:6807/33331] exists,up 29f52d62-d4c3-49f9-992f-23817f5622d2 2024-09-16T11:43:29.575 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 up in weight 1 up_from 14 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/33333,v1:172.21.15.192:6817/33333] [v2:172.21.15.192:6818/33333,v1:172.21.15.192:6819/33333] exists,up 0c1a275a-3c75-4e0b-bd35-d34b64d99c0a 2024-09-16T11:43:29.575 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6801/33332,v1:172.21.15.192:6802/33332] [v2:172.21.15.192:6804/33332,v1:172.21.15.192:6806/33332] exists,up d79f0070-ce58-48e7-a56a-2762ec76a271 2024-09-16T11:43:29.575 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7 up in weight 1 up_from 14 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/33362,v1:172.21.15.192:6825/33362] [v2:172.21.15.192:6826/33362,v1:172.21.15.192:6827/33362] exists,up 537fa5df-3d0a-4dd4-8155-20ee94e41c82 2024-09-16T11:43:29.585 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6800/33355,v1:172.21.15.70:6801/33355] [v2:172.21.15.70:6802/33355,v1:172.21.15.70:6803/33355] exists,up 91f052e4-4c8f-4484-92ce-bd1a7f8f3efb', 'osd.1 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6816/33359,v1:172.21.15.70:6817/33359] [v2:172.21.15.70:6818/33359,v1:172.21.15.70:6819/33359] exists,up 568b1280-7365-4094-afe8-b0b5e766de6f', 'osd.2 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6808/33356,v1:172.21.15.70:6809/33356] [v2:172.21.15.70:6810/33356,v1:172.21.15.70:6811/33356] exists,up 328adfa3-c1b4-4745-9759-7327c8cbd499', 'osd.3 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6824/33384,v1:172.21.15.70:6825/33384] [v2:172.21.15.70:6826/33384,v1:172.21.15.70:6827/33384] exists,up 6df66ae7-e835-4d8f-bd27-243d7a9705b5', 'osd.4 up in weight 1 up_from 13 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/33331,v1:172.21.15.192:6803/33331] [v2:172.21.15.192:6805/33331,v1:172.21.15.192:6807/33331] exists,up 29f52d62-d4c3-49f9-992f-23817f5622d2', 'osd.5 up in weight 1 up_from 14 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/33333,v1:172.21.15.192:6817/33333] [v2:172.21.15.192:6818/33333,v1:172.21.15.192:6819/33333] exists,up 0c1a275a-3c75-4e0b-bd35-d34b64d99c0a', 'osd.6 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6801/33332,v1:172.21.15.192:6802/33332] [v2:172.21.15.192:6804/33332,v1:172.21.15.192:6806/33332] exists,up d79f0070-ce58-48e7-a56a-2762ec76a271', 'osd.7 up in weight 1 up_from 14 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/33362,v1:172.21.15.192:6825/33362] [v2:172.21.15.192:6826/33362,v1:172.21.15.192:6827/33362] exists,up 537fa5df-3d0a-4dd4-8155-20ee94e41c82'] 2024-09-16T11:43:29.585 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-16T11:43:29.678 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:29.679 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mon.a", 2024-09-16T11:43:29.679 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster": "ceph", 2024-09-16T11:43:29.679 INFO:teuthology.orchestra.run.smithi070.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-16T11:43:29.679 INFO:teuthology.orchestra.run.smithi070.stdout: "admin_socket_mode": "", 2024-09-16T11:43:29.679 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-16T11:43:29.679 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_client_required": "cephx, none", 2024-09-16T11:43:29.679 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_cluster_required": "cephx", 2024-09-16T11:43:29.679 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_debug": "true", 2024-09-16T11:43:29.679 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-16T11:43:29.679 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-16T11:43:29.679 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_service_required": "cephx", 2024-09-16T11:43:29.679 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-16T11:43:29.680 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_supported": "cephx", 2024-09-16T11:43:29.680 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_aio": "true", 2024-09-16T11:43:29.680 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-16T11:43:29.680 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_aio_poll_ms": "250", 2024-09-16T11:43:29.680 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_aio_reap_max": "16", 2024-09-16T11:43:29.680 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_async_discard": "false", 2024-09-16T11:43:29.680 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_block_size": "4096", 2024-09-16T11:43:29.680 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_debug_aio": "false", 2024-09-16T11:43:29.680 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-16T11:43:29.680 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-16T11:43:29.680 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_debug_inflight_ios": "false", 2024-09-16T11:43:29.680 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_enable_discard": "false", 2024-09-16T11:43:29.680 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_flock_retry": "3", 2024-09-16T11:43:29.681 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-16T11:43:29.681 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_inject_crash": "0", 2024-09-16T11:43:29.681 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-16T11:43:29.681 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_ioring": "false", 2024-09-16T11:43:29.681 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_ioring_hipri": "false", 2024-09-16T11:43:29.681 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-16T11:43:29.681 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-16T11:43:29.681 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-16T11:43:29.681 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-16T11:43:29.681 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_type": "", 2024-09-16T11:43:29.681 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_alloc_size": "1048576", 2024-09-16T11:43:29.681 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_allocator": "hybrid", 2024-09-16T11:43:29.681 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_buffered_io": "true", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_check_for_zeros": "false", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_compact_log_sync": "false", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_max_log_runway": "4194304", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_max_prefetch": "1048576", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_min_flush_size": "524288", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_min_log_runway": "1048576", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_replay_recovery": "false", 2024-09-16T11:43:29.682 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-16T11:43:29.683 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-16T11:43:29.683 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_sync_write": "false", 2024-09-16T11:43:29.683 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-16T11:43:29.683 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-16T11:43:29.683 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-16T11:43:29.683 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_allocation_from_file": "true", 2024-09-16T11:43:29.683 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_allocator": "hybrid", 2024-09-16T11:43:29.683 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_async_db_compaction": "true", 2024-09-16T11:43:29.683 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-16T11:43:29.683 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-16T11:43:29.683 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-16T11:43:29.683 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-16T11:43:29.683 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-16T11:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-16T11:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-16T11:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_create": "true", 2024-09-16T11:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_db_create": "false", 2024-09-16T11:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_db_path": "", 2024-09-16T11:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_db_size": "0", 2024-09-16T11:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_path": "", 2024-09-16T11:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_preallocate_file": "false", 2024-09-16T11:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_size": "107374182400", 2024-09-16T11:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_wal_create": "false", 2024-09-16T11:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_wal_path": "", 2024-09-16T11:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_wal_size": "100663296", 2024-09-16T11:43:29.684 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bluefs": "true", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_autotune": "true", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-16T11:43:29.685 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_size": "0", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_type": "2q", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_clone_cow": "true", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-16T11:43:29.686 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-16T11:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_mode": "none", 2024-09-16T11:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-16T11:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_csum_type": "crc32c", 2024-09-16T11:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-16T11:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_freelist": "false", 2024-09-16T11:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-16T11:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-16T11:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-16T11:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-16T11:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-16T11:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-16T11:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-16T11:43:29.687 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_small_allocations": "0", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_default_buffered_read": "true", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_default_buffered_write": "false", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-16T11:43:29.688 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-16T11:43:29.689 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-16T11:43:29.689 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-16T11:43:29.689 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-16T11:43:29.689 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-16T11:43:29.689 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fail_eio": "false", 2024-09-16T11:43:29.689 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-16T11:43:29.689 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-16T11:43:29.689 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-16T11:43:29.689 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-16T11:43:29.689 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-16T11:43:29.689 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-16T11:43:29.689 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_on_mount": "false", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_on_umount": "false", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_ignore_data_csum": "false", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-16T11:43:29.690 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_log_op_age": "5.000000", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_max_alloc_size": "0", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_max_blob_size": "0", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_max_deferred_txc": "32", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_min_alloc_size": "0", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_nid_prealloc": "1024", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-16T11:43:29.691 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-16T11:43:29.692 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-16T11:43:29.692 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-16T11:43:29.692 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_retry_disk_reads": "3", 2024-09-16T11:43:29.692 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_rocksdb_cf": "true", 2024-09-16T11:43:29.692 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-09-16T11:43:29.692 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-09-16T11:43:29.692 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-16T11:43:29.692 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-16T11:43:29.692 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-16T11:43:29.692 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-16T11:43:29.692 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_spdk_mem": "512", 2024-09-16T11:43:29.692 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-16T11:43:29.693 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-16T11:43:29.693 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-16T11:43:29.693 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-16T11:43:29.693 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-16T11:43:29.693 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-16T11:43:29.693 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-16T11:43:29.693 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_tracing": "false", 2024-09-16T11:43:29.693 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-16T11:43:29.693 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-16T11:43:29.693 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-16T11:43:29.693 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-16T11:43:29.693 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-16T11:43:29.693 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_zero_block_detection": "false", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-16T11:43:29.694 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_cluster_require_signatures": "false", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_cluster_require_version": "2", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_require_signatures": "false", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_require_version": "2", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_service_require_signatures": "false", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_service_require_version": "2", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_sign_messages": "true", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "chdir": "", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "client_acl_type": "", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "client_asio_thread_count": "2", 2024-09-16T11:43:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: "client_cache_mid": "0.750000", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_cache_size": "16384", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_caps_release_delay": "5", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_check_pool_perm": "true", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_debug_force_sync_read": "false", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_debug_getattr_caps": "false", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_debug_inject_features": "", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_debug_inject_tick_delay": "0", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_die_on_failed_remount": "false", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_dirsize_rbytes": "true", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_force_lazyio": "false", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_fs": "", 2024-09-16T11:43:29.696 INFO:teuthology.orchestra.run.smithi070.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_inject_release_failure": "false", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_max_inline_size": "4096", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_mds_namespace": "", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_metadata": "", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_mount_gid": "-1", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_mount_timeout": "300", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_mount_uid": "-1", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_mountpoint": "/", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_notify_timeout": "10", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_oc": "true", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_oc_max_dirty": "104857600", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-16T11:43:29.697 INFO:teuthology.orchestra.run.smithi070.stdout: "client_oc_max_objects": "1000", 2024-09-16T11:43:29.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_oc_size": "209715200", 2024-09-16T11:43:29.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_oc_target_dirty": "8388608", 2024-09-16T11:43:29.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_permissions": "true", 2024-09-16T11:43:29.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_quota": "true", 2024-09-16T11:43:29.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_quota_df": "true", 2024-09-16T11:43:29.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_readahead_max_bytes": "0", 2024-09-16T11:43:29.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_readahead_max_periods": "4", 2024-09-16T11:43:29.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_readahead_min": "131072", 2024-09-16T11:43:29.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_reconnect_stale": "false", 2024-09-16T11:43:29.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_shutdown_timeout": "30", 2024-09-16T11:43:29.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_snapdir": ".snap", 2024-09-16T11:43:29.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_tick_interval": "1", 2024-09-16T11:43:29.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_trace": "", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "client_try_dentry_invalidate": "false", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "client_use_faked_inos": "false", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "client_use_random_mds": "false", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_graylog": "false", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_graylog_port": "12201", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_monitors": "default=true", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_syslog": "false", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_syslog_level": "info", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "-", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_network": "", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_network_interface": "", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "compressor_zlib_isal": "false", 2024-09-16T11:43:29.699 INFO:teuthology.orchestra.run.smithi070.stdout: "compressor_zlib_level": "5", 2024-09-16T11:43:29.700 INFO:teuthology.orchestra.run.smithi070.stdout: "compressor_zlib_winsize": "-15", 2024-09-16T11:43:29.700 INFO:teuthology.orchestra.run.smithi070.stdout: "compressor_zstd_level": "1", 2024-09-16T11:43:29.700 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-16T11:43:29.700 INFO:teuthology.orchestra.run.smithi070.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-16T11:43:29.700 INFO:teuthology.orchestra.run.smithi070.stdout: "crimson_alien_op_num_threads": "6", 2024-09-16T11:43:29.700 INFO:teuthology.orchestra.run.smithi070.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-16T11:43:29.700 INFO:teuthology.orchestra.run.smithi070.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-16T11:43:29.700 INFO:teuthology.orchestra.run.smithi070.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-16T11:43:29.700 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_location": "", 2024-09-16T11:43:29.700 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_location_hook": "", 2024-09-16T11:43:29.700 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_location_hook_timeout": "10", 2024-09-16T11:43:29.700 INFO:teuthology.orchestra.run.smithi070.stdout: "daemonize": "false", 2024-09-16T11:43:29.700 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_alienstore": "0/5", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_allow_any_pool_priority": "false", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_asok": "1/5", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_asok_assert_abort": "false", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_asserts_on_shutdown": "true", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_auth": "20/20", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_bdev": "1/3", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_bluefs": "1/5", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_bluestore": "1/5", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_buffer": "0/1", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_ceph_exporter": "1/5", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_cephfs_mirror": "0/5", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_cephsqlite": "0/5", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_civetweb": "1/10", 2024-09-16T11:43:29.701 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_client": "0/5", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_compressor": "1/5", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_context": "0/1", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_crush": "1/1", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_crypto": "1/5", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_deliberately_leak_memory": "false", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_disable_randomized_ping": "false", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_dpdk": "1/5", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_eventtrace": "1/5", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_filer": "0/1", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_filestore": "1/3", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_finisher": "1/1", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_fuse": "1/5", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_heartbeat_testing_span": "0", 2024-09-16T11:43:29.702 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_heartbeatmap": "1/5", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_javaclient": "1/5", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_journal": "1/3", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_journaler": "0/5", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_kstore": "1/5", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_leveldb": "4/5", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_lockdep": "0/1", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mclock": "1/5", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mds": "1/5", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mds_balancer": "1/5", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mds_locker": "1/5", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mds_log": "1/5", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mds_log_expire": "1/5", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mds_migrator": "1/5", 2024-09-16T11:43:29.703 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_memdb": "4/5", 2024-09-16T11:43:29.704 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mgr": "2/5", 2024-09-16T11:43:29.704 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mgrc": "1/5", 2024-09-16T11:43:29.704 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mon": "20/20", 2024-09-16T11:43:29.704 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_monc": "0/10", 2024-09-16T11:43:29.704 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_ms": "1/1", 2024-09-16T11:43:29.704 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_none": "0/5", 2024-09-16T11:43:29.704 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_objclass": "0/5", 2024-09-16T11:43:29.704 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_objectcacher": "0/5", 2024-09-16T11:43:29.704 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_objecter": "0/1", 2024-09-16T11:43:29.704 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_optracker": "0/5", 2024-09-16T11:43:29.704 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_osd": "1/5", 2024-09-16T11:43:29.704 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_paxos": "20/20", 2024-09-16T11:43:29.704 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_perfcounter": "1/5", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_prioritycache": "1/5", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rados": "0/5", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rbd": "0/5", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rbd_mirror": "0/5", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rbd_pwl": "0/5", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rbd_replay": "0/5", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_refs": "0/0", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_reserver": "1/1", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rgw": "1/5", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rgw_datacache": "1/5", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rgw_sync": "1/5", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rocksdb": "4/5", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore": "0/5", 2024-09-16T11:43:29.705 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_cache": "0/5", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_cleaner": "0/5", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_device": "0/5", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_journal": "0/5", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_lba": "0/5", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_odata": "0/5", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_omap": "0/5", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_onode": "0/5", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_tm": "0/5", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_striper": "0/1", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_test": "0/5", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_throttle": "1/1", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_timer": "0/1", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_tp": "0/5", 2024-09-16T11:43:29.706 INFO:teuthology.orchestra.run.smithi070.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "device_failure_prediction_mode": "none", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "err_to_graylog": "false", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "err_to_journald": "false", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "err_to_stderr": "true", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "err_to_syslog": "false", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "event_tracing": "false", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_addr": "0.0.0.0", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_http_port": "9926", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_prio_limit": "5", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_sort_metrics": "true", 2024-09-16T11:43:29.707 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_stats_period": "5", 2024-09-16T11:43:29.708 INFO:teuthology.orchestra.run.smithi070.stdout: "fake_statfs_for_testing": "0", 2024-09-16T11:43:29.708 INFO:teuthology.orchestra.run.smithi070.stdout: "fatal_signal_handlers": "true", 2024-09-16T11:43:29.708 INFO:teuthology.orchestra.run.smithi070.stdout: "filer_max_purge_ops": "10", 2024-09-16T11:43:29.708 INFO:teuthology.orchestra.run.smithi070.stdout: "filer_max_truncate_ops": "128", 2024-09-16T11:43:29.708 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_apply_finisher_threads": "1", 2024-09-16T11:43:29.708 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_blackhole": "false", 2024-09-16T11:43:29.708 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_btrfs_clone_range": "true", 2024-09-16T11:43:29.708 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_btrfs_snap": "true", 2024-09-16T11:43:29.708 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_caller_concurrency": "10", 2024-09-16T11:43:29.708 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_collect_device_partition_information": "true", 2024-09-16T11:43:29.708 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_commit_timeout": "600.000000", 2024-09-16T11:43:29.708 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_debug_inject_read_err": "false", 2024-09-16T11:43:29.708 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_debug_omap_check": "false", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_debug_verify_split": "false", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_dump_file": "", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fadvise": "true", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fail_eio": "true", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fd_cache_shards": "16", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fd_cache_size": "128", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fiemap": "false", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fiemap_threshold": "4096", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-16T11:43:29.709 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_inject_stall": "0", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_journal_parallel": "false", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_journal_trailing": "false", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_journal_writeahead": "false", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_kill_at": "0", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattrs": "0", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-16T11:43:29.710 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_xattr_value_size": "0", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_merge_threshold": "-10", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_odsync_write": "false", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_omap_backend": "rocksdb", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_omap_backend_path": "", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_op_thread_timeout": "60", 2024-09-16T11:43:29.711 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_op_threads": "2", 2024-09-16T11:43:29.712 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_punch_hole": "false", 2024-09-16T11:43:29.712 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-16T11:43:29.712 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-16T11:43:29.712 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-16T11:43:29.712 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-16T11:43:29.712 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-16T11:43:29.712 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-16T11:43:29.712 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-16T11:43:29.712 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-16T11:43:29.712 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-16T11:43:29.712 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_max_ops": "50", 2024-09-16T11:43:29.712 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-16T11:43:29.712 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_seek_data_hole": "false", 2024-09-16T11:43:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_sloppy_crc": "false", 2024-09-16T11:43:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-16T11:43:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_splice": "false", 2024-09-16T11:43:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_split_multiple": "2", 2024-09-16T11:43:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_split_rand_factor": "20", 2024-09-16T11:43:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_update_to": "1000", 2024-09-16T11:43:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-16T11:43:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-16T11:43:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-16T11:43:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-16T11:43:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-16T11:43:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-16T11:43:29.713 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_enable": "true", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_xfs_extsize": "false", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_zfs_snap": "false", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "fio_dir": "/tmp/fio", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "3420bcad-625f-4e15-aaff-fb4236322168", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_allow_other": "true", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_atomic_o_trunc": "true", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_big_writes": "true", 2024-09-16T11:43:29.714 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_debug": "false", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_default_permissions": "false", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_disable_pagecache": "false", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_max_write": "0", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_multithreaded": "true", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_require_active_mds": "true", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_set_user_groups": "true", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_splice_move": "true", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_splice_read": "true", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_splice_write": "true", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_use_invalidate_cb": "true", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "gss_target_name": "ceph", 2024-09-16T11:43:29.715 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_file": "", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_inject_failure": "0", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_interval": "5", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "host": "smithi070", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-16T11:43:29.716 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-16T11:43:29.717 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-16T11:43:29.717 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-16T11:43:29.717 INFO:teuthology.orchestra.run.smithi070.stdout: "inject_early_sigterm": "false", 2024-09-16T11:43:29.717 INFO:teuthology.orchestra.run.smithi070.stdout: "jaeger_tracing_enable": "false", 2024-09-16T11:43:29.717 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_aio": "true", 2024-09-16T11:43:29.717 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_align_min_size": "65536", 2024-09-16T11:43:29.717 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_block_align": "true", 2024-09-16T11:43:29.717 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_block_size": "4096", 2024-09-16T11:43:29.717 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_dio": "true", 2024-09-16T11:43:29.717 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_discard": "false", 2024-09-16T11:43:29.717 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_force_aio": "false", 2024-09-16T11:43:29.717 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_ignore_corruption": "false", 2024-09-16T11:43:29.717 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_max_write_bytes": "10485760", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_max_write_entries": "100", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_replay_from": "0", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_write_header_frequency": "0", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_zero_on_create": "false", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "journaler_prefetch_periods": "10", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "journaler_prezero_periods": "5", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "journaler_write_head_interval": "15", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "keyfile": "", 2024-09-16T11:43:29.718 INFO:teuthology.orchestra.run.smithi070.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_backend": "rocksdb", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_default_stripe_size": "65536", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_fsck_on_mount": "false", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_max_bytes": "67108864", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_max_ops": "512", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_nid_prealloc": "1024", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_onode_map_size": "1024", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_sync_submit_transaction": "false", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_sync_transaction": "false", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "leveldb_block_size": "65536", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "leveldb_bloom_size": "0", 2024-09-16T11:43:29.719 INFO:teuthology.orchestra.run.smithi070.stdout: "leveldb_cache_size": "536870912", 2024-09-16T11:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout: "leveldb_compact_on_mount": "false", 2024-09-16T11:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout: "leveldb_compression": "false", 2024-09-16T11:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout: "leveldb_log": "", 2024-09-16T11:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-16T11:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout: "leveldb_max_open_files": "0", 2024-09-16T11:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout: "leveldb_paranoid": "false", 2024-09-16T11:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-16T11:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout: "librados_thread_count": "2", 2024-09-16T11:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout: "lockdep": "false", 2024-09-16T11:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout: "lockdep_force_backtrace": "false", 2024-09-16T11:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout: "log_coarse_timestamps": "true", 2024-09-16T11:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-16T11:43:29.720 INFO:teuthology.orchestra.run.smithi070.stdout: "log_flush_on_exit": "false", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "log_graylog_host": "127.0.0.1", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "log_graylog_port": "12201", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "log_max_new": "1000", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "log_max_recent": "10000", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "log_stderr_prefix": "", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "log_stop_at_utilization": "0.970000", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": "true", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_graylog": "false", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_journald": "false", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_stderr": "false", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_syslog": "false", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "max_rotating_auth_attempts": "10", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-16T11:43:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_action_on_write_error": "1", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_alternate_name_max": "8192", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_asio_thread_count": "2", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_export_pin": "true", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_fragment_dirs": "true", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_fragment_interval": "5", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_interval": "10", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_max": "-1", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_max_until": "-1", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_merge_size": "50", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_midchunk": "0.300000", 2024-09-16T11:43:29.722 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_min_start": "0.200000", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_minchunk": "0.001000", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_mode": "0", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_need_max": "1.200000", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_need_min": "0.800000", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_split_bits": "3", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_split_size": "10000", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_target_decay": "10.000000", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-16T11:43:29.723 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_beacon_grace": "15.000000", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_beacon_interval": "4.000000", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_mid": "0.700000", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_release_free_interval": "10", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_reservation": "0.050000", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_trim_interval": "1", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_trim_threshold": "262144", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_client_prealloc_inos": "1000", 2024-09-16T11:43:29.724 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_connect_bootstrapping": "false", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_damage_table_max_entries": "10000", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_debug_auth_pins": "false", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_debug_frag": "false", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_debug_scatterstat": "false", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_debug_subtrees": "false", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_decay_halflife": "5.000000", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_default_dir_hash": "2", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_defer_session_stale": "true", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_deny_all_reconnect": "false", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dir_keys_per_op": "16384", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dir_max_commit_size": "10", 2024-09-16T11:43:29.725 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dir_max_entries": "0", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dump_cache_on_map": "false", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_early_reply": "true", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_enable_op_tracker": "true", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_enforce_unique_name": "true", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_export_ephemeral_random": "true", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-16T11:43:29.726 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_health_summarize_threshold": "10", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_inject_health_dummy": "false", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_inject_migrator_session_race": "false", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_join_fs": "", 2024-09-16T11:43:29.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_journal_format": "1", 2024-09-16T11:43:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_create_at": "0", 2024-09-16T11:43:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_export_at": "0", 2024-09-16T11:43:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_import_at": "0", 2024-09-16T11:43:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_journal_at": "0", 2024-09-16T11:43:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_journal_expire_at": "0", 2024-09-16T11:43:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_journal_replay_at": "0", 2024-09-16T11:43:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_link_at": "0", 2024-09-16T11:43:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_mdstable_at": "0", 2024-09-16T11:43:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_openc_at": "0", 2024-09-16T11:43:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_rename_at": "0", 2024-09-16T11:43:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-09-16T11:43:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_events_per_segment": "1024", 2024-09-16T11:43:29.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_max_events": "-1", 2024-09-16T11:43:29.729 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_max_segments": "128", 2024-09-16T11:43:29.729 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_pause": "false", 2024-09-16T11:43:29.729 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_segment_size": "0", 2024-09-16T11:43:29.729 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-16T11:43:29.729 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_warn_factor": "2.000000", 2024-09-16T11:43:29.729 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_caps_per_client": "1048576", 2024-09-16T11:43:29.729 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_completed_flushes": "100000", 2024-09-16T11:43:29.729 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_completed_requests": "100000", 2024-09-16T11:43:29.729 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_export_size": "20971520", 2024-09-16T11:43:29.729 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_file_recover": "32", 2024-09-16T11:43:29.729 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_purge_files": "64", 2024-09-16T11:43:29.729 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_purge_ops": "8192", 2024-09-16T11:43:29.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-16T11:43:29.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-16T11:43:29.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_snaps_per_dir": "100", 2024-09-16T11:43:29.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_xattr_pairs_size": "65536", 2024-09-16T11:43:29.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_metrics_update_interval": "2", 2024-09-16T11:43:29.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_min_caps_per_client": "100", 2024-09-16T11:43:29.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_min_caps_working_set": "10000", 2024-09-16T11:43:29.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-16T11:43:29.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_numa_node": "-1", 2024-09-16T11:43:29.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-16T11:43:29.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_op_complaint_time": "30.000000", 2024-09-16T11:43:29.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_op_history_duration": "600", 2024-09-16T11:43:29.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_op_history_size": "20", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_op_history_slow_op_size": "20", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_op_log_threshold": "5", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_ping_grace": "15", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_ping_interval": "5", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_recall_max_caps": "30000", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_recall_warning_threshold": "262144", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_replay_interval": "1.000000", 2024-09-16T11:43:29.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-16T11:43:29.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-16T11:43:29.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_root_ino_gid": "0", 2024-09-16T11:43:29.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_root_ino_uid": "0", 2024-09-16T11:43:29.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-16T11:43:29.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-16T11:43:29.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-16T11:43:29.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-16T11:43:29.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-16T11:43:29.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-16T11:43:29.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-16T11:43:29.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-16T11:43:29.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-16T11:43:29.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-16T11:43:29.733 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_shutdown_check": "0", 2024-09-16T11:43:29.733 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_skip_ino": "0", 2024-09-16T11:43:29.733 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-16T11:43:29.733 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_snap_max_uid": "4294967294", 2024-09-16T11:43:29.733 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_snap_min_uid": "0", 2024-09-16T11:43:29.733 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_snap_rstat": "false", 2024-09-16T11:43:29.733 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_standby_replay_damaged": "false", 2024-09-16T11:43:29.733 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_symlink_recovery": "true", 2024-09-16T11:43:29.733 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-16T11:43:29.733 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_thrash_exports": "0", 2024-09-16T11:43:29.733 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_thrash_fragments": "0", 2024-09-16T11:43:29.733 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_tick_interval": "5.000000", 2024-09-16T11:43:29.733 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_valgrind_exit": "false", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_verify_backtrace": "1", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_verify_scatter": "false", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_wipe_sessions": "false", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool_debug": "false", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "memstore_device_bytes": "1073741824", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "memstore_page_set": "false", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "memstore_page_size": "65536", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_client_bytes": "134217728", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_client_messages": "512", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-16T11:43:29.734 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_disabled_modules": "diskprediction_local", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_max_pg_num_change": "128", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_mds_bytes": "134217728", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_mds_messages": "128", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_mon_bytes": "134217728", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_mon_messages": "128", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_osd_bytes": "536870912", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_osd_messages": "8192", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_pool": "true", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-16T11:43:29.735 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_standby_modules": "true", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_stats_period": "5", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_stats_threshold": "5", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_tick_period": "2", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_allow_pool_delete": "true", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_allow_pool_size_one": "true", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_auth_validate_all_caps": "true", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_bytes": "104857600", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_directed_command_retry": "5", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-16T11:43:29.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_hunt_parallel": "3", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_log_interval": "1.000000", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_ping_interval": "10.000000", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_level": "debug", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_file": "true", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-16T11:43:29.737 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_journald": "false", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_compact_on_bootstrap": "false", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_compact_on_start": "false", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_compact_on_trim": "true", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cpu_threads": "4", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_crush_min_required_version": "hammer", 2024-09-16T11:43:29.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_daemon_bytes": "419430400", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_data_avail_crit": "5", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_data_avail_warn": "5", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_data_size_warn": "16106127360", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_dump_json": "false", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_dump_transactions": "false", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_extra_checks": "false", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_no_require_pacific": "false", 2024-09-16T11:43:29.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_no_require_quincy": "false", 2024-09-16T11:43:29.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-16T11:43:29.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-16T11:43:29.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-16T11:43:29.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_down_mkfs_grace": "120", 2024-09-16T11:43:29.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_election_default_strategy": "1", 2024-09-16T11:43:29.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_election_timeout": "5.000000", 2024-09-16T11:43:29.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-16T11:43:29.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_elector_ping_divisor": "2", 2024-09-16T11:43:29.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-16T11:43:29.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_enable_op_tracker": "true", 2024-09-16T11:43:29.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_fake_pool_delete": "false", 2024-09-16T11:43:29.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_force_quorum_join": "false", 2024-09-16T11:43:29.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_globalid_prealloc": "10000", 2024-09-16T11:43:29.741 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_health_detail_to_clog": "false", 2024-09-16T11:43:29.741 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_health_log_update_period": "5", 2024-09-16T11:43:29.741 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_health_max_detail": "50", 2024-09-16T11:43:29.741 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_health_to_clog": "true", 2024-09-16T11:43:29.741 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_health_to_clog_interval": "600", 2024-09-16T11:43:29.741 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-16T11:43:29.741 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_host": "172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790],172.21.15.192", 2024-09-16T11:43:29.741 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_host_override": "", 2024-09-16T11:43:29.741 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_initial_members": "", 2024-09-16T11:43:29.741 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-16T11:43:29.741 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-16T11:43:29.741 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-16T11:43:29.741 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_lease": "5.000000", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_log_full_interval": "50", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_log_max": "10000", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_log_max_summary": "50", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_log_epochs": "500", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_osd": "10000", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_pg_per_osd": "10000", 2024-09-16T11:43:29.742 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_pool_pg_num": "65536", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mds_force_trim_to": "0", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mds_skip_sanity": "false", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_memory_autotune": "true", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_memory_target": "2147483648", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mgr_beacon_grace": "30", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mgr_digest_period": "5", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mgr_inactive_grace": "60", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_min_osdmap_epochs": "500", 2024-09-16T11:43:29.743 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_op_complaint_time": "30", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_op_history_duration": "600", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_op_history_size": "20", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_op_history_slow_op_size": "20", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_op_log_threshold": "5", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_auto_mark_in": "false", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_cache_size": "500", 2024-09-16T11:43:29.744 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_down_out_interval": "600", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_force_trim_to": "0", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-16T11:43:29.745 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-16T11:43:29.746 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_min_down_reporters": "2", 2024-09-16T11:43:29.746 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-16T11:43:29.746 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-16T11:43:29.746 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-16T11:43:29.746 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-16T11:43:29.746 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-16T11:43:29.746 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-16T11:43:29.746 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_report_timeout": "900", 2024-09-16T11:43:29.746 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-16T11:43:29.746 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-16T11:43:29.746 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-16T11:43:29.746 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-16T11:43:29.746 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pg_stuck_threshold": "60", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_probe_timeout": "2.000000", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_reweight_max_change": "0.050000", 2024-09-16T11:43:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_reweight_max_osds": "4", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_scrub_interval": "86400", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_scrub_max_keys": "100", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_scrub_timeout": "300", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_session_timeout": "300", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_smart_report_timeout": "5", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_stat_smooth_intervals": "6", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_stretch_pool_min_size": "2", 2024-09-16T11:43:29.748 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_stretch_pool_size": "4", 2024-09-16T11:43:29.749 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-16T11:43:29.749 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-16T11:43:29.752 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_sync_debug": "false", 2024-09-16T11:43:29.752 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-16T11:43:29.752 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-16T11:43:29.752 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_sync_provider_kill_at": "0", 2024-09-16T11:43:29.752 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_sync_requester_kill_at": "0", 2024-09-16T11:43:29.752 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_sync_timeout": "60.000000", 2024-09-16T11:43:29.752 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_target_pg_per_osd": "100", 2024-09-16T11:43:29.752 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_tick_interval": "5", 2024-09-16T11:43:29.752 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_timecheck_interval": "300.000000", 2024-09-16T11:43:29.753 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-16T11:43:29.753 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_older_version_delay": "604800", 2024-09-16T11:43:29.753 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-16T11:43:29.753 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-16T11:43:29.753 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-16T11:43:29.753 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-16T11:43:29.753 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-16T11:43:29.753 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-16T11:43:29.753 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-16T11:43:29.753 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_misplaced": "false", 2024-09-16T11:43:29.753 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-16T11:43:29.753 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_older_version": "true", 2024-09-16T11:43:29.753 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "monmap": "", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_op_threads": "3", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_cm": "false", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_device_name": "", 2024-09-16T11:43:29.754 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_dscp": "96", 2024-09-16T11:43:29.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-16T11:43:29.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-16T11:43:29.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_local_gid": "", 2024-09-16T11:43:29.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-16T11:43:29.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_port_num": "1", 2024-09-16T11:43:29.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-16T11:43:29.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-16T11:43:29.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-16T11:43:29.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-16T11:43:29.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_sl": "3", 2024-09-16T11:43:29.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_support_srq": "true", 2024-09-16T11:43:29.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_type": "ib", 2024-09-16T11:43:29.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_reap_threshold": "5", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_before_connect": "false", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_ipv4": "true", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_ipv6": "false", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_msgr1": "true", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_msgr2": "true", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_port_max": "7568", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_port_min": "6800", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_retry_count": "3", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_retry_delay": "5", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_blackhole_client": "false", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_blackhole_mds": "false", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_blackhole_mgr": "false", 2024-09-16T11:43:29.756 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_blackhole_mon": "false", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_blackhole_osd": "false", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_client_mode": "crc secure", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_cluster_mode": "crc secure", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_cluster_type": "", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_compress_secure": "false", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_connection_idle_timeout": "900", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_connection_ready_timeout": "10", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_crc_data": "true", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_crc_header": "true", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_die_on_bad_msg": "false", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_die_on_bug": "true", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_die_on_old_message": "true", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_die_on_skipped_message": "false", 2024-09-16T11:43:29.757 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-16T11:43:29.758 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-16T11:43:29.758 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_coremask": "0xF", 2024-09-16T11:43:29.758 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-16T11:43:29.758 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-16T11:43:29.758 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_enable_tso": "true", 2024-09-16T11:43:29.758 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-16T11:43:29.758 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-16T11:43:29.759 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_hugepages": "", 2024-09-16T11:43:29.759 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-16T11:43:29.759 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-16T11:43:29.759 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_lro": "true", 2024-09-16T11:43:29.759 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_memory_channel": "4", 2024-09-16T11:43:29.759 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-16T11:43:29.759 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_pmd": "", 2024-09-16T11:43:29.759 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_port_id": "0", 2024-09-16T11:43:29.759 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-16T11:43:29.759 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-16T11:43:29.759 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dump_on_send": "false", 2024-09-16T11:43:29.759 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_initial_backoff": "0.200000", 2024-09-16T11:43:29.759 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_inject_delay_max": "1.000000", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_inject_delay_type": "", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_inject_socket_failures": "5000", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_learn_addr_from_peer": "true", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_max_accept_failures": "4", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_max_backoff": "15.000000", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_mon_client_mode": "secure crc", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_mon_service_mode": "secure crc", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_osd_compress_min_size": "1024", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_osd_compress_mode": "none", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2024-09-16T11:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-16T11:43:29.761 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_pq_min_cost": "65536", 2024-09-16T11:43:29.761 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_public_type": "", 2024-09-16T11:43:29.761 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_service_mode": "crc secure", 2024-09-16T11:43:29.761 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_tcp_listen_backlog": "512", 2024-09-16T11:43:29.761 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_tcp_nodelay": "true", 2024-09-16T11:43:29.761 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-16T11:43:29.761 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_tcp_rcvbuf": "0", 2024-09-16T11:43:29.761 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_type": "async+posix", 2024-09-16T11:43:29.761 INFO:teuthology.orchestra.run.smithi070.stdout: "no_config_file": "false", 2024-09-16T11:43:29.761 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_completion_locks_per_session": "32", 2024-09-16T11:43:29.761 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-16T11:43:29.761 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-16T11:43:29.761 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_inflight_ops": "1024", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_tick_interval": "5.000000", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_timeout": "10.000000", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "objectstore_blackhole": "false", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "openssl_engine_opts": "", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_delay_time": "5.000000", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_hist_halflife": "1000", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_max_low_ops": "2", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_max_ops": "4", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_slop": "0.020000", 2024-09-16T11:43:29.762 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_asio_thread_count": "2", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_async_recovery_min_cost": "100", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_backfill_scan_max": "512", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_backfill_scan_min": "64", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_backoff_on_degraded": "false", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_backoff_on_peering": "false", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_backoff_on_unfound": "true", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_beacon_report_interval": "300", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bench_duration": "30", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-16T11:43:29.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bench_max_block_size": "67108864", 2024-09-16T11:43:29.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-16T11:43:29.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_blkin_trace_all": "false", 2024-09-16T11:43:29.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-16T11:43:29.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-16T11:43:29.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_check_for_log_corruption": "false", 2024-09-16T11:43:29.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-16T11:43:29.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T11:43:29.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-16T11:43:29.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T11:43:29.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_class_update_on_start": "true", 2024-09-16T11:43:29.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_client_message_cap": "256", 2024-09-16T11:43:29.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_client_message_size_cap": "524288000", 2024-09-16T11:43:29.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_client_op_priority": "63", 2024-09-16T11:43:29.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_client_watch_timeout": "30", 2024-09-16T11:43:29.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_command_max_records": "256", 2024-09-16T11:43:29.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_compact_on_start": "false", 2024-09-16T11:43:29.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-16T11:43:29.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-16T11:43:29.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-16T11:43:29.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_crush_update_on_start": "true", 2024-09-16T11:43:29.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_crush_update_weight_set": "true", 2024-09-16T11:43:29.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-16T11:43:29.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-16T11:43:29.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-16T11:43:29.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-16T11:43:29.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_feed_pullee": "-1", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_misdirected_ops": "false", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_no_acting_change": "false", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_no_purge_strays": "false", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_op_order": "false", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_shutdown": "false", 2024-09-16T11:43:29.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_verify_snaps": "false", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_keys": "1024", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_stride": "524288", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-16T11:43:29.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_default_notify_timeout": "30", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_delete_sleep": "0.000000", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_discard_disconnected_ops": "true", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_enable_op_tracker": "true", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_fast_info": "true", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_fast_shutdown": "true", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-16T11:43:29.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-16T11:43:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-16T11:43:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-16T11:43:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_function_tracing": "false", 2024-09-16T11:43:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_grace": "20", 2024-09-16T11:43:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_interval": "6", 2024-09-16T11:43:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-16T11:43:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_min_peers": "10", 2024-09-16T11:43:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_min_size": "2000", 2024-09-16T11:43:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_stale": "600", 2024-09-16T11:43:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-16T11:43:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_hit_set_max_size": "100000", 2024-09-16T11:43:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_hit_set_min_size": "1000", 2024-09-16T11:43:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-16T11:43:29.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-16T11:43:29.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-16T11:43:29.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-16T11:43:29.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-16T11:43:29.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-16T11:43:29.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_journal_size": "5120", 2024-09-16T11:43:29.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_kill_backfill_at": "0", 2024-09-16T11:43:29.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-16T11:43:29.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_map_cache_size": "50", 2024-09-16T11:43:29.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_map_dedup": "true", 2024-09-16T11:43:29.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_map_message_max": "40", 2024-09-16T11:43:29.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-16T11:43:29.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_map_share_max_epochs": "40", 2024-09-16T11:43:29.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_attr_name_len": "100", 2024-09-16T11:43:29.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_attr_size": "0", 2024-09-16T11:43:29.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_backfills": "1", 2024-09-16T11:43:29.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_markdown_count": "5", 2024-09-16T11:43:29.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_markdown_period": "600", 2024-09-16T11:43:29.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_object_name_len": "2048", 2024-09-16T11:43:29.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_object_namespace_len": "256", 2024-09-16T11:43:29.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_object_size": "134217728", 2024-09-16T11:43:29.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-16T11:43:29.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-16T11:43:29.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_pg_blocked_by": "16", 2024-09-16T11:43:29.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_pg_log_entries": "10000", 2024-09-16T11:43:29.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_pgls": "1024", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_push_cost": "8388608", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_push_objects": "10", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_scrubs": "3", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_trimming_pgs": "2", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_write_op_reply_len": "64", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_write_size": "90", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-16T11:43:29.772 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-16T11:43:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-16T11:43:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-16T11:43:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_profile": "balanced", 2024-09-16T11:43:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-16T11:43:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-16T11:43:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-16T11:43:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-16T11:43:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-16T11:43:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-16T11:43:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-16T11:43:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-16T11:43:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-16T11:43:29.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-16T11:43:29.774 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-16T11:43:29.774 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_base": "805306368", 2024-09-16T11:43:29.774 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_cache_min": "134217728", 2024-09-16T11:43:29.774 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-16T11:43:29.774 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-16T11:43:29.774 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_target": "4294967296", 2024-09-16T11:43:29.774 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_target_autotune": "false", 2024-09-16T11:43:29.774 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-16T11:43:29.774 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_min_pg_log_entries": "250", 2024-09-16T11:43:29.774 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_min_recovery_priority": "0", 2024-09-16T11:43:29.774 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-16T11:43:29.774 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-16T11:43:29.774 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mon_report_interval": "5", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_num_cache_shards": "32", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_num_op_tracker_shard": "32", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_numa_auto_affinity": "true", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_numa_node": "-1", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_numa_prefer_iface": "true", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_objecter_finishers": "1", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_objectstore": "bluestore", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_objectstore_fuse": "false", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_objectstore_tracing": "false", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_complaint_time": "30.000000", 2024-09-16T11:43:29.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_history_duration": "600", 2024-09-16T11:43:29.776 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_history_size": "20", 2024-09-16T11:43:29.776 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_history_slow_op_size": "20", 2024-09-16T11:43:29.776 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-16T11:43:29.776 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_log_threshold": "5", 2024-09-16T11:43:29.776 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_num_shards": "0", 2024-09-16T11:43:29.776 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_num_shards_hdd": "5", 2024-09-16T11:43:29.776 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_num_shards_ssd": "8", 2024-09-16T11:43:29.776 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-16T11:43:29.776 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-16T11:43:29.776 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-16T11:43:29.776 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-16T11:43:29.776 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_pq_min_cost": "65536", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_queue_cut_off": "high", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_thread_timeout": "15", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_open_classes_on_start": "true", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_os_flags": "0", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_peering_op_priority": "255", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_delete_cost": "1048576", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_delete_priority": "5", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_log_trim_max": "10000", 2024-09-16T11:43:29.777 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_log_trim_min": "100", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-16T11:43:29.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_flags": "0", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_min_size": "0", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_pg_num": "32", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_pgp_num": "0", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_size": "2", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_type": "replicated", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-16T11:43:29.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-16T11:43:29.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_push_per_object_cost": "1000", 2024-09-16T11:43:29.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-16T11:43:29.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recover_clone_overlap": "true", 2024-09-16T11:43:29.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-16T11:43:29.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_cost": "20971520", 2024-09-16T11:43:29.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-16T11:43:29.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_max_active": "0", 2024-09-16T11:43:29.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-16T11:43:29.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-16T11:43:29.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-16T11:43:29.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-16T11:43:29.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_max_single_start": "1", 2024-09-16T11:43:29.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_op_priority": "3", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_priority": "5", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_sleep": "0.000000", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_repair_during_recovery": "false", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_requested_scrub_priority": "120", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_auto_repair": "false", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-16T11:43:29.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_begin_hour": "0", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_begin_week_day": "0", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_chunk_max": "25", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_chunk_min": "5", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_cost": "52428800", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_during_recovery": "false", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_end_hour": "0", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_end_week_day": "0", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_event_cost": "4096", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_invalid_stats": "true", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-16T11:43:29.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_max_preemptions": "5", 2024-09-16T11:43:29.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-16T11:43:29.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_priority": "5", 2024-09-16T11:43:29.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_sleep": "0.000000", 2024-09-16T11:43:29.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-16T11:43:29.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_skip_data_digest": "false", 2024-09-16T11:43:29.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_smart_report_timeout": "5", 2024-09-16T11:43:29.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_snap_trim_cost": "1048576", 2024-09-16T11:43:29.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_snap_trim_priority": "5", 2024-09-16T11:43:29.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-16T11:43:29.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-16T11:43:29.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-16T11:43:29.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-16T11:43:29.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_target_transaction_size": "30", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tracing": "false", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_use_stale_snap": "false", 2024-09-16T11:43:29.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "osdc_blkin_trace_all": "false", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_kill_at": "0", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_max_join_drift": "10", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_min": "500", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_min_wait": "0.050000", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_propose_interval": "1.000000", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_service_trim_max": "500", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_service_trim_min": "250", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_stash_full_interval": "25", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_trim_max": "500", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_trim_min": "250", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "perf": "true", 2024-09-16T11:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "-", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrv": "", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "public_bind_addr": "-", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "public_network": "", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "public_network_interface": "", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "qat_compressor_enabled": "false", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "rados_mon_op_timeout": "0", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "rados_osd_op_timeout": "0", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "rados_tracing": "false", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_atime_update_interval": "60", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-16T11:43:29.786 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_balance_parent_reads": "false", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_balance_snap_reads": "false", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_blkin_trace_all": "false", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache": "true", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_policy": "writearound", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_size": "33554432", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-16T11:43:29.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_clone_copy_on_read": "false", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_compression_hint": "none", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_concurrent_management_ops": "10", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_clone_format": "auto", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_data_pool": "", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_features": "61", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_format": "2", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_map_options": "", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_order": "22", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_pool": "rbd", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_stripe_count": "0", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_stripe_unit": "0", 2024-09-16T11:43:29.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_enable_alloc_hint": "true", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_io_scheduler": "simple", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-16T11:43:29.789 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_order": "24", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_pool": "", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_splay_width": "4", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_localize_parent_reads": "false", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_localize_snap_reads": "false", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-16T11:43:29.790 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-16T11:43:29.791 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mtime_update_interval": "60", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_non_blocking_aio": "true", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_op_thread_timeout": "60", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_op_threads": "1", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_parent_cache_enabled": "false", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-16T11:43:29.792 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_plugins": "", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_bps_burst": "0", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_bps_limit": "0", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_exclude_ops": "0", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_iops_burst": "0", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_iops_limit": "0", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-16T11:43:29.793 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-16T11:43:29.794 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-16T11:43:29.794 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-16T11:43:29.794 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-16T11:43:29.794 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-16T11:43:29.794 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-16T11:43:29.794 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-16T11:43:29.794 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-16T11:43:29.794 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-16T11:43:29.794 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-16T11:43:29.794 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_read_from_replica_policy": "default", 2024-09-16T11:43:29.794 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_skip_partial_discard": "true", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_tracing": "false", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_validate_names": "true", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_validate_pool": "true", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "restapi_base_url": "", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "restapi_log_level": "", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_acl_grants_max_num": "100", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_admin_entry": "admin", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_backend_store": "rados", 2024-09-16T11:43:29.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_barbican_url": "", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_beast_enable_async": "true", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_cache_enabled": "true", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_cache_expiry_interval": "900", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_cache_lru_size": "10000", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_content_length_compat": "false", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_copy_obj_progress": "true", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_cors_rules_max_num": "100", 2024-09-16T11:43:29.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_cross_domain_policy": "", 2024-09-16T11:43:29.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-16T11:43:29.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_addr": "", 2024-09-16T11:43:29.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-16T11:43:29.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-16T11:43:29.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-16T11:43:29.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-16T11:43:29.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_password": "", 2024-09-16T11:43:29.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-16T11:43:29.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_username": "", 2024-09-16T11:43:29.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_require_ssl": "true", 2024-09-16T11:43:29.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-16T11:43:29.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-16T11:43:29.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-16T11:43:29.798 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-16T11:43:29.798 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-16T11:43:29.798 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-16T11:43:29.798 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-16T11:43:29.798 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-16T11:43:29.798 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-16T11:43:29.798 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-16T11:43:29.798 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-16T11:43:29.798 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-16T11:43:29.798 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-16T11:43:29.798 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-16T11:43:29.798 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-16T11:43:29.798 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_addr": "", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_auth": "token", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_namespace": "", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_prefix": "", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_token_file": "", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_curl_buffersize": "524288", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_curl_low_speed_time": "300", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-16T11:43:29.799 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data_log_changes_size": "1000", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data_log_num_shards": "128", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data_log_window": "30", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data_notify_interval_msec": "200", 2024-09-16T11:43:29.800 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-16T11:43:29.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dns_name": "", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dns_s3website_name": "", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dynamic_resharding": "true", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_gc_threads": "true", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_lc_threads": "true", 2024-09-16T11:43:29.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_ops_log": "false", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_quota_threads": "true", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_static_website": "false", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_usage_log": "false", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enforce_swift_acls": "true", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_exit_timeout_secs": "120", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_expose_bucket": "false", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_extended_http_attrs": "", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_frontends": "beast port=7480", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_max_deferred": "50", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_max_objs": "32", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-16T11:43:29.803 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-16T11:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-16T11:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-16T11:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_processor_period": "3600", 2024-09-16T11:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-16T11:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-16T11:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-16T11:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-16T11:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_init_timeout": "300", 2024-09-16T11:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-16T11:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_kafka_connection_idle": "300", 2024-09-16T11:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-16T11:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-16T11:43:29.804 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_domain": "", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_password": "", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_password_path": "", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_project": "", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_tenant": "", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_token": "", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_token_path": "", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_user": "", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_api_version": "2", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_barbican_domain": "", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_barbican_password": "", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_barbican_project": "", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_barbican_user": "", 2024-09-16T11:43:29.805 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_url": "", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_debug_interval": "-1", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_lock_max_time": "90", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_max_objs": "32", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_max_rules": "1000", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_max_worker": "3", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_thread_delay": "0", 2024-09-16T11:43:29.806 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ldap_dnattr": "uid", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ldap_searchfilter": "", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ldap_uri": "", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_log_http_headers": "", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_log_object_name_utc": "false", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-16T11:43:29.807 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_attr_name_len": "0", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_attr_size": "0", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_chunk_size": "4194304", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_listing_results": "1000", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_notify_retries": "10", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_put_param_size": "1048576", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_put_size": "5368709120", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_slo_entries": "1000", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_md_log_max_shards": "64", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-16T11:43:29.808 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_mp_lock_max_time": "600", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_max_gc": "300", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-16T11:43:29.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_num_async_rados_threads": "32", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_num_control_oids": "8", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_numa_node": "-1", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_objexp_chunk_size": "100", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_objexp_gc_interval": "600", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_op_thread_timeout": "600", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_op_tracing": "false", 2024-09-16T11:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_opa_token": "", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_opa_url": "", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_opa_verify_ssl": "true", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ops_log_file_path": "", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ops_log_rados": "true", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ops_log_socket_path": "", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_period_push_interval": "2.000000", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_print_continue": "true", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-16T11:43:29.811 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_rados_tracing": "false", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_realm": "", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_realm_id": "", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_region": "", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_request_uri": "", 2024-09-16T11:43:29.812 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshard_batch_size": "64", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshard_max_aio": "128", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshard_num_logs": "16", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshard_thread_interval": "600", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_resolve_cname": "false", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_run_sync_thread": "true", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-16T11:43:29.813 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-16T11:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-16T11:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-16T11:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_scheduler_type": "throttler", 2024-09-16T11:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_script_uri": "", 2024-09-16T11:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_service_provider_name": "", 2024-09-16T11:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-16T11:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_client_id": "", 2024-09-16T11:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_client_secret": "", 2024-09-16T11:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_entry": "sts", 2024-09-16T11:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_key": "sts", 2024-09-16T11:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-16T11:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_min_session_duration": "900", 2024-09-16T11:43:29.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_token_introspection_url": "", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_account_in_url": "false", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_auth_entry": "auth", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_auth_url": "", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_custom_header": "", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_need_stats": "true", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_tenant_name": "", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_token_expiration": "86400", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_url": "", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_url_prefix": "swift", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_lease_period": "120", 2024-09-16T11:43:29.815 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_thread_pool_size": "512", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_comment": "", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_createby": "", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_encoding": "", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_flag": "false", 2024-09-16T11:43:29.816 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_origin": "", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_tracker": "", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_trust_forwarded_https": "false", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_usage_max_shards": "32", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_usage_max_user_shards": "1", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_use_opa_authz": "false", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_max_buckets": "1000", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-16T11:43:29.817 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-16T11:43:29.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-16T11:43:29.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_unique_email": "true", 2024-09-16T11:43:29.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_verify_ssl": "true", 2024-09-16T11:43:29.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-16T11:43:29.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_zone": "", 2024-09-16T11:43:29.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_zone_id": "", 2024-09-16T11:43:29.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-16T11:43:29.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_zonegroup": "", 2024-09-16T11:43:29.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_zonegroup_id": "", 2024-09-16T11:43:29.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-16T11:43:29.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_block_size": "4096", 2024-09-16T11:43:29.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-16T11:43:29.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cache_size": "536870912", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_index_type": "binary_search", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-16T11:43:29.819 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-16T11:43:29.820 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_partition_filters": "false", 2024-09-16T11:43:29.820 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_perf": "false", 2024-09-16T11:43:29.820 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-16T11:43:29.820 INFO:teuthology.orchestra.run.smithi070.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-16T11:43:29.820 INFO:teuthology.orchestra.run.smithi070.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-16T11:43:29.820 INFO:teuthology.orchestra.run.smithi070.stdout: "run_dir": "/var/run/ceph", 2024-09-16T11:43:29.820 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_block_create": "true", 2024-09-16T11:43:29.820 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_cache_lru_size": "67108864", 2024-09-16T11:43:29.820 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_default_max_object_size": "16777216", 2024-09-16T11:43:29.820 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-16T11:43:29.820 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_device_size": "107374182400", 2024-09-16T11:43:29.820 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_init_rewrite_segments_num_per_device": "6", 2024-09-16T11:43:29.820 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_journal_batch_capacity": "16", 2024-09-16T11:43:29.821 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-16T11:43:29.821 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-16T11:43:29.821 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-16T11:43:29.821 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_segment_size": "67108864", 2024-09-16T11:43:29.821 INFO:teuthology.orchestra.run.smithi070.stdout: "setgroup": "", 2024-09-16T11:43:29.821 INFO:teuthology.orchestra.run.smithi070.stdout: "setuser": "", 2024-09-16T11:43:29.821 INFO:teuthology.orchestra.run.smithi070.stdout: "setuser_match_path": "", 2024-09-16T11:43:29.821 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-16T11:43:29.821 INFO:teuthology.orchestra.run.smithi070.stdout: "thp": "false", 2024-09-16T11:43:29.821 INFO:teuthology.orchestra.run.smithi070.stdout: "threadpool_default_timeout": "60", 2024-09-16T11:43:29.821 INFO:teuthology.orchestra.run.smithi070.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-16T11:43:29.821 INFO:teuthology.orchestra.run.smithi070.stdout: "throttler_perf_counter": "true" 2024-09-16T11:43:29.821 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:29.823 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-16T11:43:29.930 INFO:teuthology.orchestra.run.smithi070.stdout:mon.a: {} 2024-09-16T11:43:29.930 INFO:teuthology.orchestra.run.smithi070.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T11:43:29.939 INFO:teuthology.orchestra.run.smithi070.stdout:mon.b: {} 2024-09-16T11:43:29.939 INFO:teuthology.orchestra.run.smithi070.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T11:43:29.947 INFO:teuthology.orchestra.run.smithi070.stdout:mon.c: {} 2024-09-16T11:43:29.947 INFO:teuthology.orchestra.run.smithi070.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T11:43:29.957 DEBUG:teuthology.orchestra.run.smithi192:> type ceph-objectstore-tool 2024-09-16T11:43:29.975 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T11:43:29.975 DEBUG:teuthology.orchestra.run.smithi070:> type ceph-objectstore-tool 2024-09-16T11:43:29.990 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T11:43:29.990 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-16T11:43:30.000 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-16T11:43:30.001 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-16T11:43:30.001 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:43:30.001 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:43:30.001 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-16T11:43:30.001 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:43:30.001 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:43:30.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-16T11:43:30.002 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-16T11:43:30.002 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-16T11:43:30.002 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:43:30.002 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-16T11:43:30.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:43:30.003 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-16T11:43:30.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:43:30.003 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-16T11:43:30.017 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:30.015+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:30.063 INFO:teuthology.orchestra.run.smithi070.stdout:mapped writes DISABLED 2024-09-16T11:43:30.063 INFO:teuthology.orchestra.run.smithi070.stdout:mapped reads DISABLED 2024-09-16T11:43:30.063 INFO:teuthology.orchestra.run.smithi070.stdout:Seed set to 8345 2024-09-16T11:43:30.104 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:30.102+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:30.141 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:30.141 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:30.141 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:30.141 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:30.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:43:30.155 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:30.156 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T11:43:30.156 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:30.157 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:43:30.157 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:43:30.174 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:43:30.175 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:43:30.190 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:43:30.190 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:43:30.205 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:30.204+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:30.206 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:43:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:43:30.223 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:43:30.223 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:43:30.239 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:43:30.239 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:43:30.254 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:30.254 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:30.255 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:43:30.255 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:43:30.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:43:30.272 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:43:30.273 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:43:30.324 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:30.322+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:30.369 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:30.370 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:30.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:43:30.407 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:30.405+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:30.508 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:30.505+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:30.516 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:30.517 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:30.525 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:43:30.609 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:30.607+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:30.622 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:30.622 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:30.631 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:43:30.710 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:30.708+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:30.729 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:30.729 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:30.738 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:43:30.764 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T11:43:30.820 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:30.818+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:30.863 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:30.864 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:30.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:43:30.912 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:30.911+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:31.010 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:31.010 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:31.013 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:31.011+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:31.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:43:31.114 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:31.112+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:31.156 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:31.156 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:31.165 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:43:31.215 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:31.213+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:31.303 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:31.303 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:31.311 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:43:31.316 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:31.314+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:31.409 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:31.409 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:31.416 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:31.415+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:31.417 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:43:31.517 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:31.517 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:31.517 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:31.515+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:31.525 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:43:31.618 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:31.615+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:31.635 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:31.635 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:31.644 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:43:31.718 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:31.716+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:31.789 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:31.789 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:31.798 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:43:31.844 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:31.842+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:31.899 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:31.899 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:31.907 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:43:31.920 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:31.918+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:32.021 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:32.019+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:32.048 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:32.048 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:32.057 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:43:32.122 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:32.119+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:32.154 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:32.155 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:32.163 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:43:32.247 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:32.246+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:32.284 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:43:32.302 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:32.302 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:32.310 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:43:32.352 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:32.350+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:32.409 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:43:32.409 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:43:32.424 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:43:32.424 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:43:32.425 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:32.422+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:32.439 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:43:32.439 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:43:32.449 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:32.449 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:32.453 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:43:32.453 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:43:32.457 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:43:32.468 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:43:32.468 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:43:32.484 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:43:32.484 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:43:32.499 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:43:32.499 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:43:32.514 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:43:32.514 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:43:32.525 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:32.524+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:32.557 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:32.557 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:32.557 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:32.557 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:32.557 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:32.565 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:43:32.626 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:32.623+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:32.706 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:32.706 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:43:32.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:43:32.706 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-16T11:43:32.706 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:32.714 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:43:32.726 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:32.724+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:32.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:43:32.815 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:32.816 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:32.816 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:43:32.816 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:32.824 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:43:32.827 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:32.824+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:32.922 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:32.922 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:32.922 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:32.922 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:32.923 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:32.928 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:32.926+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:32.930 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:43:33.030 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:43:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:43:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:43:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.718083+0000", 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.308677118999999, 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061853209999999997, 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718083+0000", 2024-09-16T11:43:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718083+0000", 2024-09-16T11:43:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718084+0000", 2024-09-16T11:43:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-16T11:43:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718153+0000", 2024-09-16T11:43:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9362000000000004e-05 2024-09-16T11:43:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718159+0000", 2024-09-16T11:43:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3060000000000003e-06 2024-09-16T11:43:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718164+0000", 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3900000000000003e-06 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719004+0000", 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00084073899999999996 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719050+0000", 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5124999999999999e-05 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719125+0000", 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5914999999999994e-05 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720731+0000", 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016052589999999999 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724213+0000", 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034819180000000001 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724245+0000", 2024-09-16T11:43:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2725999999999998e-05 2024-09-16T11:43:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724268+0000", 2024-09-16T11:43:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2954000000000001e-05 2024-09-16T11:43:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.718169+0000", 2024-09-16T11:43:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.308591575000001, 2024-09-16T11:43:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061430139999999996, 2024-09-16T11:43:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 23 2024-09-16T11:43:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718169+0000", 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718169+0000", 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718169+0000", 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718227+0000", 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7714999999999999e-05 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718233+0000", 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9320000000000001e-06 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718236+0000", 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:43:33.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719452+0000", 2024-09-16T11:43:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001215673 2024-09-16T11:43:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719501+0000", 2024-09-16T11:43:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9283000000000002e-05 2024-09-16T11:43:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719584+0000", 2024-09-16T11:43:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3182000000000004e-05 2024-09-16T11:43:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720759+0000", 2024-09-16T11:43:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011749810000000001 2024-09-16T11:43:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724273+0000", 2024-09-16T11:43:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035134649999999999 2024-09-16T11:43:33.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724294+0000", 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1158999999999999e-05 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724312+0000", 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740999999999998e-05 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.718240+0000", 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.30852028, 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061176770000000002, 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 24 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718240+0000", 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718240+0000", 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718240+0000", 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718296+0000", 2024-09-16T11:43:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5365999999999998e-05 2024-09-16T11:43:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718301+0000", 2024-09-16T11:43:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5219999999999998e-06 2024-09-16T11:43:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718308+0000", 2024-09-16T11:43:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5440000000000004e-06 2024-09-16T11:43:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719993+0000", 2024-09-16T11:43:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001685717 2024-09-16T11:43:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720044+0000", 2024-09-16T11:43:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1007000000000001e-05 2024-09-16T11:43:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720123+0000", 2024-09-16T11:43:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8714999999999994e-05 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720918+0000", 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00079541099999999999 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724319+0000", 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034004740000000001 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724340+0000", 2024-09-16T11:43:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1131000000000001e-05 2024-09-16T11:43:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724358+0000", 2024-09-16T11:43:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7458000000000001e-05 2024-09-16T11:43:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.718312+0000", 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.308448565999999, 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060983740000000002, 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718312+0000", 2024-09-16T11:43:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718312+0000", 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718312+0000", 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718314+0000", 2024-09-16T11:43:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718318+0000", 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3970000000000004e-06 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718321+0000", 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4219999999999999e-06 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720425+0000", 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021042690000000002 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720474+0000", 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8449000000000003e-05 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720548+0000", 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4276999999999994e-05 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.721085+0000", 2024-09-16T11:43:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000537252 2024-09-16T11:43:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724361+0000", 2024-09-16T11:43:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0032759289999999999 2024-09-16T11:43:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724393+0000", 2024-09-16T11:43:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1930000000000001e-05 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724410+0000", 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7002000000000001e-05 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.919904+0000", 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.106856582999999, 2024-09-16T11:43:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047939870000000004, 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.919904+0000", 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.919904+0000", 2024-09-16T11:43:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.919904+0000", 2024-09-16T11:43:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0299999999999999e-07 2024-09-16T11:43:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.919968+0000", 2024-09-16T11:43:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3954999999999993e-05 2024-09-16T11:43:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.919990+0000", 2024-09-16T11:43:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2368999999999999e-05 2024-09-16T11:43:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.919996+0000", 2024-09-16T11:43:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3940000000000004e-06 2024-09-16T11:43:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.922480+0000", 2024-09-16T11:43:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024846019999999998 2024-09-16T11:43:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.922542+0000", 2024-09-16T11:43:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1821000000000003e-05 2024-09-16T11:43:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.922634+0000", 2024-09-16T11:43:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1349999999999998e-05 2024-09-16T11:43:33.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924048+0000", 2024-09-16T11:43:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014146810000000001 2024-09-16T11:43:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924645+0000", 2024-09-16T11:43:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00059679700000000004 2024-09-16T11:43:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924676+0000", 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1081000000000003e-05 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924698+0000", 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1433999999999998e-05 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920009+0000", 2024-09-16T11:43:33.060 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.106750671, 2024-09-16T11:43:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048056660000000001, 2024-09-16T11:43:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-16T11:43:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920009+0000", 2024-09-16T11:43:33.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920009+0000", 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920010+0000", 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920013+0000", 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7e-06 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920017+0000", 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999997e-06 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920021+0000", 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7440000000000001e-06 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923078+0000", 2024-09-16T11:43:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030574170000000002 2024-09-16T11:43:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923130+0000", 2024-09-16T11:43:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1388999999999999e-05 2024-09-16T11:43:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923210+0000", 2024-09-16T11:43:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9909999999999999e-05 2024-09-16T11:43:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924710+0000", 2024-09-16T11:43:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015001940000000001 2024-09-16T11:43:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924756+0000", 2024-09-16T11:43:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5726999999999998e-05 2024-09-16T11:43:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924784+0000", 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8822e-05 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924815+0000", 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0775999999999998e-05 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:43:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.106736415, 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:43:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:43:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:43:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:43:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.106690723, 2024-09-16T11:43:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:43:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:43:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:43:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:43:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:43:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:43:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:43:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:43:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:43:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:43:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:43:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:43:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:43:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:43:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:43:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:43:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:43:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:43:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:43:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:43:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:43:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:43:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:43:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:43:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:43:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:43:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.078 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.106633793, 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:43:33.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:43:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:43:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:43:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:43:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:43:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:43:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:43:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:43:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:43:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:43:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:43:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:43:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:43:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:43:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:43:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:43:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:43:33.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:43:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:43:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:43:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.106608425000001, 2024-09-16T11:43:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:43:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:43:33.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:43:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:43:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:43:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:43:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:43:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:43:33.090 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:43:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.106556457, 2024-09-16T11:43:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:43:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:43:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:43:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:43:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:43:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:43:33.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:43:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:43:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:43:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:43:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:43:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:43:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:43:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:43:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:43:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:43:33.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:43:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:43:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.106475347, 2024-09-16T11:43:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:43:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:43:33.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:43:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:43:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:43:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:43:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:43:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:43:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:43:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:43:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:43:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:43:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:43:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:43:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:43:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:43:33.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:43:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:43:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:43:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.103 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.106449369, 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:43:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:43:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:43:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:43:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:43:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:43:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:43:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:43:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:43:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:43:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:43:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:43:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:43:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:43:33.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:43:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:43:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:43:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.109 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337703+0000", 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.689056701999998, 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053663890000000001, 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 70 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337703+0000", 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337703+0000", 2024-09-16T11:43:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337704+0000", 2024-09-16T11:43:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:43:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337706+0000", 2024-09-16T11:43:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:43:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337711+0000", 2024-09-16T11:43:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2900000000000002e-06 2024-09-16T11:43:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337714+0000", 2024-09-16T11:43:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3970000000000001e-06 2024-09-16T11:43:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341010+0000", 2024-09-16T11:43:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0032962220000000001 2024-09-16T11:43:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341092+0000", 2024-09-16T11:43:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1996999999999994e-05 2024-09-16T11:43:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341220+0000", 2024-09-16T11:43:33.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012852600000000001 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.342994+0000", 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001773279 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343021+0000", 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.758e-05 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343049+0000", 2024-09-16T11:43:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8118000000000001e-05 2024-09-16T11:43:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343070+0000", 2024-09-16T11:43:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0330999999999999e-05 2024-09-16T11:43:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.115 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337716+0000", 2024-09-16T11:43:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.689043952999999, 2024-09-16T11:43:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054005590000000001, 2024-09-16T11:43:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 71 2024-09-16T11:43:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337716+0000", 2024-09-16T11:43:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337716+0000", 2024-09-16T11:43:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337716+0000", 2024-09-16T11:43:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-16T11:43:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337718+0000", 2024-09-16T11:43:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.657e-06 2024-09-16T11:43:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337723+0000", 2024-09-16T11:43:33.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3329999999999996e-06 2024-09-16T11:43:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337726+0000", 2024-09-16T11:43:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.039e-06 2024-09-16T11:43:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341844+0000", 2024-09-16T11:43:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041171699999999999 2024-09-16T11:43:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341901+0000", 2024-09-16T11:43:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7717999999999999e-05 2024-09-16T11:43:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341974+0000", 2024-09-16T11:43:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2538000000000002e-05 2024-09-16T11:43:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343017+0000", 2024-09-16T11:43:33.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010429510000000001 2024-09-16T11:43:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343079+0000", 2024-09-16T11:43:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2045000000000007e-05 2024-09-16T11:43:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343100+0000", 2024-09-16T11:43:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1154000000000001e-05 2024-09-16T11:43:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343117+0000", 2024-09-16T11:43:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6741000000000001e-05 2024-09-16T11:43:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337730+0000", 2024-09-16T11:43:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.68903001, 2024-09-16T11:43:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063357409999999998, 2024-09-16T11:43:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-16T11:43:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337730+0000", 2024-09-16T11:43:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337730+0000", 2024-09-16T11:43:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337730+0000", 2024-09-16T11:43:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-16T11:43:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337785+0000", 2024-09-16T11:43:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4673000000000003e-05 2024-09-16T11:43:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337794+0000", 2024-09-16T11:43:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8829999999999995e-06 2024-09-16T11:43:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337798+0000", 2024-09-16T11:43:33.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1069999999999998e-06 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.342223+0000", 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.004425216 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.342309+0000", 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5424000000000001e-05 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.342451+0000", 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014200899999999999 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343980+0000", 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001529819 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344021+0000", 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0175000000000001e-05 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344046+0000", 2024-09-16T11:43:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5677000000000002e-05 2024-09-16T11:43:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344066+0000", 2024-09-16T11:43:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9551999999999998e-05 2024-09-16T11:43:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.128 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337806+0000", 2024-09-16T11:43:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.688954134999999, 2024-09-16T11:43:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.006307857, 2024-09-16T11:43:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-16T11:43:33.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337806+0000", 2024-09-16T11:43:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337806+0000", 2024-09-16T11:43:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337807+0000", 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7199999999999999e-07 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337812+0000", 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7259999999999999e-06 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337820+0000", 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9079999999999995e-06 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337824+0000", 2024-09-16T11:43:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0740000000000003e-06 2024-09-16T11:43:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343154+0000", 2024-09-16T11:43:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053298640000000001 2024-09-16T11:43:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343203+0000", 2024-09-16T11:43:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8664e-05 2024-09-16T11:43:33.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343284+0000", 2024-09-16T11:43:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1211000000000006e-05 2024-09-16T11:43:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344015+0000", 2024-09-16T11:43:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00073082900000000001 2024-09-16T11:43:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344074+0000", 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9416000000000002e-05 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344095+0000", 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0837999999999999e-05 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344114+0000", 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8754999999999999e-05 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.688930744, 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:43:33.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:43:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:43:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:43:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:43:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:43:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:43:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:43:33.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:43:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:43:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:43:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:43:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:43:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:43:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.688908813000001, 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:43:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:43:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:43:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:43:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:43:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:43:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:43:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:43:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:43:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:43:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:43:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:43:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:43:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:43:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:43:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:43:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:43:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:43:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:43:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.688831924999999, 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:43:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:43:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:43:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:43:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:43:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:43:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:43:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:43:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:43:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:43:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:43:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:43:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:43:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:43:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:43:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:43:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:33.153 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:33.153 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:33.028+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:33.154 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:43:33.196 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:33.194+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:33.253 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:33.253 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:33.253 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:33.253 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:33.256 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:33.254+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:33.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:43:33.357 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:33.355+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:33.399 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:33.399 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:33.399 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:33.399 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:33.400 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:33.408 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:43:33.492 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:33.490+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:33.547 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:33.548 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:33.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:33.548 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:33.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.548 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:21 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'11, mlcod=15'8)", 2024-09-16T11:43:33.548 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:12.718741+0000", 2024-09-16T11:43:33.548 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.826204334, 2024-09-16T11:43:33.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00080984000000000004, 2024-09-16T11:43:33.548 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.548 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.548 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.718741+0000", 2024-09-16T11:43:33.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.718741+0000", 2024-09-16T11:43:33.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.718742+0000", 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5400000000000001e-07 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.718812+0000", 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9956999999999994e-05 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.718823+0000", 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1144999999999999e-05 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.718828+0000", 2024-09-16T11:43:33.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.587e-06 2024-09-16T11:43:33.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.718859+0000", 2024-09-16T11:43:33.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1210000000000001e-05 2024-09-16T11:43:33.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.718871+0000", 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1746000000000001e-05 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.719529+0000", 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00065742399999999999 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.719551+0000", 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2217000000000001e-05 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.552 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:22 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'12, mlcod=15'8)", 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:12.719237+0000", 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.825708195000001, 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00075962000000000002, 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.719237+0000", 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.719237+0000", 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.719238+0000", 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8299999999999997e-07 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.719306+0000", 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8220999999999996e-05 2024-09-16T11:43:33.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.719317+0000", 2024-09-16T11:43:33.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1419e-05 2024-09-16T11:43:33.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.719323+0000", 2024-09-16T11:43:33.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2970000000000003e-06 2024-09-16T11:43:33.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.719352+0000", 2024-09-16T11:43:33.556 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9158000000000001e-05 2024-09-16T11:43:33.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.556 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.556 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.719363+0000", 2024-09-16T11:43:33.556 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1061e-05 2024-09-16T11:43:33.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.556 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.556 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.719978+0000", 2024-09-16T11:43:33.556 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00061555400000000004 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.719997+0000", 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8326999999999999e-05 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:34 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'20, mlcod=15'18)", 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:12.917227+0000", 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.627718776999998, 2024-09-16T11:43:33.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00076835599999999999, 2024-09-16T11:43:33.558 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.558 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.558 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.917227+0000", 2024-09-16T11:43:33.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.917227+0000", 2024-09-16T11:43:33.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.917231+0000", 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.019e-06 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.917243+0000", 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2238999999999999e-05 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.917257+0000", 2024-09-16T11:43:33.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3831e-05 2024-09-16T11:43:33.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.560 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.917270+0000", 2024-09-16T11:43:33.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3182e-05 2024-09-16T11:43:33.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.560 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.917322+0000", 2024-09-16T11:43:33.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1858999999999999e-05 2024-09-16T11:43:33.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.561 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.917395+0000", 2024-09-16T11:43:33.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2806999999999996e-05 2024-09-16T11:43:33.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.561 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.917959+0000", 2024-09-16T11:43:33.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00056463100000000001 2024-09-16T11:43:33.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.917995+0000", 2024-09-16T11:43:33.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5787999999999997e-05 2024-09-16T11:43:33.562 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.562 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.562 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.562 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:39 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'19)", 2024-09-16T11:43:33.562 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:12.919847+0000", 2024-09-16T11:43:33.562 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.625098641000001, 2024-09-16T11:43:33.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00086593499999999997, 2024-09-16T11:43:33.562 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.562 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.563 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.563 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.919847+0000", 2024-09-16T11:43:33.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.563 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.919847+0000", 2024-09-16T11:43:33.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.919847+0000", 2024-09-16T11:43:33.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2300000000000001e-07 2024-09-16T11:43:33.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.919857+0000", 2024-09-16T11:43:33.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0125999999999999e-05 2024-09-16T11:43:33.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.919867+0000", 2024-09-16T11:43:33.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6209999999999999e-06 2024-09-16T11:43:33.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.919874+0000", 2024-09-16T11:43:33.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9319999999999999e-06 2024-09-16T11:43:33.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.919910+0000", 2024-09-16T11:43:33.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5599000000000003e-05 2024-09-16T11:43:33.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.919933+0000", 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3215000000000001e-05 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.920685+0000", 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00075259399999999998 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.920713+0000", 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7325e-05 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.566 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.567 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.567 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:41 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'19)", 2024-09-16T11:43:33.567 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:12.920854+0000", 2024-09-16T11:43:33.567 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.624091505999999, 2024-09-16T11:43:33.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00087828199999999998, 2024-09-16T11:43:33.567 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.567 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.567 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.567 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.920854+0000", 2024-09-16T11:43:33.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.920854+0000", 2024-09-16T11:43:33.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.920854+0000", 2024-09-16T11:43:33.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.15e-07 2024-09-16T11:43:33.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.920930+0000", 2024-09-16T11:43:33.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6118000000000005e-05 2024-09-16T11:43:33.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.920938+0000", 2024-09-16T11:43:33.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7719999999999994e-06 2024-09-16T11:43:33.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.920947+0000", 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5499999999999995e-06 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.920975+0000", 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8705e-05 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.920993+0000", 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7974000000000001e-05 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.921708+0000", 2024-09-16T11:43:33.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00071436000000000004 2024-09-16T11:43:33.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.921732+0000", 2024-09-16T11:43:33.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4488e-05 2024-09-16T11:43:33.571 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.571 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.571 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.571 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:44 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'23)", 2024-09-16T11:43:33.572 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:12.922756+0000", 2024-09-16T11:43:33.572 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.622189245000001, 2024-09-16T11:43:33.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00078359100000000004, 2024-09-16T11:43:33.572 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.572 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.572 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.572 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.572 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.922756+0000", 2024-09-16T11:43:33.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.572 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.922756+0000", 2024-09-16T11:43:33.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.922756+0000", 2024-09-16T11:43:33.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:43:33.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.922828+0000", 2024-09-16T11:43:33.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1768999999999997e-05 2024-09-16T11:43:33.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.922835+0000", 2024-09-16T11:43:33.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.55e-06 2024-09-16T11:43:33.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.922838+0000", 2024-09-16T11:43:33.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6979999999999998e-06 2024-09-16T11:43:33.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.922856+0000", 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8712e-05 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.922867+0000", 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0695e-05 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.923512+0000", 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00064522600000000005 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.923540+0000", 2024-09-16T11:43:33.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7577999999999999e-05 2024-09-16T11:43:33.576 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.576 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.576 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.576 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:45 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'27, mlcod=15'23)", 2024-09-16T11:43:33.576 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:12.923314+0000", 2024-09-16T11:43:33.576 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.621631679, 2024-09-16T11:43:33.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00073797200000000004, 2024-09-16T11:43:33.576 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.577 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.577 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.577 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.577 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.923314+0000", 2024-09-16T11:43:33.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.577 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.577 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.923314+0000", 2024-09-16T11:43:33.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.923314+0000", 2024-09-16T11:43:33.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1199999999999998e-07 2024-09-16T11:43:33.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.923318+0000", 2024-09-16T11:43:33.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9369999999999997e-06 2024-09-16T11:43:33.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.923324+0000", 2024-09-16T11:43:33.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3940000000000001e-06 2024-09-16T11:43:33.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.923329+0000", 2024-09-16T11:43:33.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1640000000000003e-06 2024-09-16T11:43:33.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.923351+0000", 2024-09-16T11:43:33.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2634e-05 2024-09-16T11:43:33.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.923361+0000", 2024-09-16T11:43:33.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0068000000000001e-05 2024-09-16T11:43:33.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.924026+0000", 2024-09-16T11:43:33.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066456800000000004 2024-09-16T11:43:33.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.581 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.581 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.924052+0000", 2024-09-16T11:43:33.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5795e-05 2024-09-16T11:43:33.581 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.581 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.582 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.582 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:49 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'31, mlcod=15'26)", 2024-09-16T11:43:33.582 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:12.925486+0000", 2024-09-16T11:43:33.582 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.619459541000001, 2024-09-16T11:43:33.582 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000852014, 2024-09-16T11:43:33.582 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.582 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.583 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925486+0000", 2024-09-16T11:43:33.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925486+0000", 2024-09-16T11:43:33.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925486+0000", 2024-09-16T11:43:33.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0699999999999998e-07 2024-09-16T11:43:33.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.584 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925490+0000", 2024-09-16T11:43:33.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1919999999999998e-06 2024-09-16T11:43:33.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.584 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925498+0000", 2024-09-16T11:43:33.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1629999999999999e-06 2024-09-16T11:43:33.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.585 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925501+0000", 2024-09-16T11:43:33.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6909999999999999e-06 2024-09-16T11:43:33.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.585 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925528+0000", 2024-09-16T11:43:33.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6488000000000001e-05 2024-09-16T11:43:33.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.586 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925538+0000", 2024-09-16T11:43:33.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0587e-05 2024-09-16T11:43:33.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.586 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926312+0000", 2024-09-16T11:43:33.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00077374799999999999 2024-09-16T11:43:33.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.586 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926338+0000", 2024-09-16T11:43:33.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5738e-05 2024-09-16T11:43:33.587 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.587 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.587 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.587 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:50 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'32, mlcod=15'26)", 2024-09-16T11:43:33.587 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:12.925917+0000", 2024-09-16T11:43:33.587 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.619028283999999, 2024-09-16T11:43:33.587 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00098522200000000001, 2024-09-16T11:43:33.587 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.587 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.587 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925917+0000", 2024-09-16T11:43:33.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925917+0000", 2024-09-16T11:43:33.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925917+0000", 2024-09-16T11:43:33.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:43:33.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925987+0000", 2024-09-16T11:43:33.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9498000000000004e-05 2024-09-16T11:43:33.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925994+0000", 2024-09-16T11:43:33.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9399999999999996e-06 2024-09-16T11:43:33.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.925999+0000", 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0679999999999996e-06 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926030+0000", 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1480999999999999e-05 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926050+0000", 2024-09-16T11:43:33.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9085000000000001e-05 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926872+0000", 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00082230499999999995 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926902+0000", 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0471e-05 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.592 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:51 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'26)", 2024-09-16T11:43:33.592 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:12.926481+0000", 2024-09-16T11:43:33.592 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.618464617000001, 2024-09-16T11:43:33.592 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00081808399999999996, 2024-09-16T11:43:33.592 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.592 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.592 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.592 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.592 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926481+0000", 2024-09-16T11:43:33.592 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.592 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.593 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.593 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926481+0000", 2024-09-16T11:43:33.593 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.593 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.593 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926481+0000", 2024-09-16T11:43:33.593 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4499999999999997e-07 2024-09-16T11:43:33.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.593 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.594 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926485+0000", 2024-09-16T11:43:33.594 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.194e-06 2024-09-16T11:43:33.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.594 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.594 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.594 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926495+0000", 2024-09-16T11:43:33.594 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3309999999999993e-06 2024-09-16T11:43:33.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.594 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.594 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.594 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926499+0000", 2024-09-16T11:43:33.594 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6210000000000003e-06 2024-09-16T11:43:33.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926532+0000", 2024-09-16T11:43:33.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2396e-05 2024-09-16T11:43:33.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.926546+0000", 2024-09-16T11:43:33.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4592e-05 2024-09-16T11:43:33.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.927268+0000", 2024-09-16T11:43:33.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00072184400000000002 2024-09-16T11:43:33.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.927299+0000", 2024-09-16T11:43:33.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0660999999999997e-05 2024-09-16T11:43:33.596 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.596 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.596 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.596 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:55 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'37, mlcod=15'33)", 2024-09-16T11:43:33.596 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:12.931968+0000", 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.612977295, 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00089673599999999997, 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.931968+0000", 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.931968+0000", 2024-09-16T11:43:33.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.598 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.598 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.931969+0000", 2024-09-16T11:43:33.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:43:33.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.598 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.598 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.932047+0000", 2024-09-16T11:43:33.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8196999999999997e-05 2024-09-16T11:43:33.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.598 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.932056+0000", 2024-09-16T11:43:33.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9760000000000001e-06 2024-09-16T11:43:33.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.599 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.932061+0000", 2024-09-16T11:43:33.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6640000000000003e-06 2024-09-16T11:43:33.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.599 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.932087+0000", 2024-09-16T11:43:33.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5477999999999999e-05 2024-09-16T11:43:33.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.932105+0000", 2024-09-16T11:43:33.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8227e-05 2024-09-16T11:43:33.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.932835+0000", 2024-09-16T11:43:33.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00073017999999999998 2024-09-16T11:43:33.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:12.932865+0000", 2024-09-16T11:43:33.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9471999999999998e-05 2024-09-16T11:43:33.601 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.601 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.601 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.601 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:66 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'44, mlcod=15'40)", 2024-09-16T11:43:33.601 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:13.338493+0000", 2024-09-16T11:43:33.601 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.206452015, 2024-09-16T11:43:33.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00075710000000000003, 2024-09-16T11:43:33.601 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.601 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.601 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.338493+0000", 2024-09-16T11:43:33.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.338493+0000", 2024-09-16T11:43:33.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.338494+0000", 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.63e-07 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.338595+0000", 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000101186 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.338602+0000", 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0169999999999999e-06 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.338607+0000", 2024-09-16T11:43:33.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5159999999999996e-06 2024-09-16T11:43:33.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.338629+0000", 2024-09-16T11:43:33.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2188000000000002e-05 2024-09-16T11:43:33.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.338640+0000", 2024-09-16T11:43:33.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0781000000000001e-05 2024-09-16T11:43:33.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.605 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.605 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.339232+0000", 2024-09-16T11:43:33.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00059287800000000002 2024-09-16T11:43:33.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.605 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.605 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.339250+0000", 2024-09-16T11:43:33.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8071e-05 2024-09-16T11:43:33.605 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:75 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'53, mlcod=15'50)", 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:13.344369+0000", 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.200575898, 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00073519700000000004, 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.606 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.344369+0000", 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.344369+0000", 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.344370+0000", 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.607 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.608 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.344419+0000", 2024-09-16T11:43:33.608 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9236000000000001e-05 2024-09-16T11:43:33.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.608 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.608 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.344428+0000", 2024-09-16T11:43:33.608 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6810000000000004e-06 2024-09-16T11:43:33.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.608 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.608 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.344432+0000", 2024-09-16T11:43:33.608 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8369999999999999e-06 2024-09-16T11:43:33.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.609 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.609 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.344457+0000", 2024-09-16T11:43:33.609 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5236e-05 2024-09-16T11:43:33.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.609 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.609 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.344468+0000", 2024-09-16T11:43:33.609 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0824e-05 2024-09-16T11:43:33.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.609 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.610 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.345076+0000", 2024-09-16T11:43:33.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00060775599999999999 2024-09-16T11:43:33.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.610 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.610 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.345105+0000", 2024-09-16T11:43:33.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9087000000000001e-05 2024-09-16T11:43:33.610 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.610 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.610 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.610 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:77 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'55, mlcod=15'53)", 2024-09-16T11:43:33.611 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:13.347014+0000", 2024-09-16T11:43:33.611 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.197931586999999, 2024-09-16T11:43:33.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00084919900000000003, 2024-09-16T11:43:33.611 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.611 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.611 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.611 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.611 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347014+0000", 2024-09-16T11:43:33.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.611 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347014+0000", 2024-09-16T11:43:33.612 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.612 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347014+0000", 2024-09-16T11:43:33.612 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8100000000000002e-07 2024-09-16T11:43:33.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.612 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347066+0000", 2024-09-16T11:43:33.612 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1208999999999997e-05 2024-09-16T11:43:33.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.613 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.613 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347076+0000", 2024-09-16T11:43:33.613 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0207000000000001e-05 2024-09-16T11:43:33.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.613 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.613 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347080+0000", 2024-09-16T11:43:33.613 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:43:33.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.613 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.613 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347118+0000", 2024-09-16T11:43:33.614 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7839000000000001e-05 2024-09-16T11:43:33.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.614 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.614 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.614 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347131+0000", 2024-09-16T11:43:33.614 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3472000000000001e-05 2024-09-16T11:43:33.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.614 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.614 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.614 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347828+0000", 2024-09-16T11:43:33.614 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00069670599999999995 2024-09-16T11:43:33.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.615 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347863+0000", 2024-09-16T11:43:33.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5172000000000001e-05 2024-09-16T11:43:33.615 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.615 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.615 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.615 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:78 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'56, mlcod=15'53)", 2024-09-16T11:43:33.615 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:13.347461+0000", 2024-09-16T11:43:33.615 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.197484165999999, 2024-09-16T11:43:33.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00081701500000000004, 2024-09-16T11:43:33.616 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.616 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.616 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347461+0000", 2024-09-16T11:43:33.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347461+0000", 2024-09-16T11:43:33.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.617 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347462+0000", 2024-09-16T11:43:33.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:43:33.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.617 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347525+0000", 2024-09-16T11:43:33.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3053999999999996e-05 2024-09-16T11:43:33.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.618 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.618 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347535+0000", 2024-09-16T11:43:33.618 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0077e-05 2024-09-16T11:43:33.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.618 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.618 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347543+0000", 2024-09-16T11:43:33.618 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8790000000000003e-06 2024-09-16T11:43:33.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.618 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.618 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347566+0000", 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3209000000000001e-05 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347579+0000", 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2959000000000001e-05 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.348253+0000", 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00067447999999999998 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.620 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.348278+0000", 2024-09-16T11:43:33.620 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4831e-05 2024-09-16T11:43:33.620 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.620 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.620 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.620 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:79 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'57, mlcod=15'53)", 2024-09-16T11:43:33.620 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:13.347908+0000", 2024-09-16T11:43:33.620 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.197037341000001, 2024-09-16T11:43:33.620 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000805995, 2024-09-16T11:43:33.620 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.621 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.621 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.621 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.621 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347908+0000", 2024-09-16T11:43:33.621 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.621 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.621 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347908+0000", 2024-09-16T11:43:33.621 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347908+0000", 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5400000000000002e-07 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347976+0000", 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7322000000000004e-05 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347984+0000", 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8849999999999999e-06 2024-09-16T11:43:33.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.623 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.623 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.347987+0000", 2024-09-16T11:43:33.623 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2590000000000001e-06 2024-09-16T11:43:33.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.623 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.623 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.348022+0000", 2024-09-16T11:43:33.623 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5209000000000001e-05 2024-09-16T11:43:33.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.623 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.623 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.348034+0000", 2024-09-16T11:43:33.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2057e-05 2024-09-16T11:43:33.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.624 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.624 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.624 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.348690+0000", 2024-09-16T11:43:33.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000655862 2024-09-16T11:43:33.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.624 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.624 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.624 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.348714+0000", 2024-09-16T11:43:33.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3947e-05 2024-09-16T11:43:33.624 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.625 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.625 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.625 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.625 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.625 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:89 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'63, mlcod=15'60)", 2024-09-16T11:43:33.625 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:13.799907+0000", 2024-09-16T11:43:33.625 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.745038271999999, 2024-09-16T11:43:33.625 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001018499, 2024-09-16T11:43:33.625 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.625 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.625 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.625 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.625 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.626 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.799907+0000", 2024-09-16T11:43:33.626 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.626 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.626 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.626 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.799907+0000", 2024-09-16T11:43:33.626 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.626 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.626 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.626 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.799908+0000", 2024-09-16T11:43:33.626 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5199999999999997e-07 2024-09-16T11:43:33.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.627 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.627 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.799915+0000", 2024-09-16T11:43:33.627 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:43:33.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.627 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.627 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.799925+0000", 2024-09-16T11:43:33.627 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0487e-05 2024-09-16T11:43:33.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.627 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.627 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.799930+0000", 2024-09-16T11:43:33.628 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0490000000000003e-06 2024-09-16T11:43:33.628 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.628 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.628 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.628 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.799954+0000", 2024-09-16T11:43:33.628 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4009999999999999e-05 2024-09-16T11:43:33.628 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.628 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.628 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.628 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.799964+0000", 2024-09-16T11:43:33.628 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9699999999999994e-06 2024-09-16T11:43:33.628 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.628 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.629 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.629 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.800892+0000", 2024-09-16T11:43:33.629 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00092748799999999997 2024-09-16T11:43:33.629 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.629 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.629 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.629 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.800926+0000", 2024-09-16T11:43:33.629 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4017999999999999e-05 2024-09-16T11:43:33.629 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.629 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.629 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.629 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.629 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:90 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'64, mlcod=15'60)", 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:13.800364+0000", 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.744581191999998, 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010375510000000001, 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.800364+0000", 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.630 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.631 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.800364+0000", 2024-09-16T11:43:33.631 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.631 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.631 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.631 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.800365+0000", 2024-09-16T11:43:33.631 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1500000000000004e-07 2024-09-16T11:43:33.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.631 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.631 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.631 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.800426+0000", 2024-09-16T11:43:33.631 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1049000000000005e-05 2024-09-16T11:43:33.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.800437+0000", 2024-09-16T11:43:33.632 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1644000000000001e-05 2024-09-16T11:43:33.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.800443+0000", 2024-09-16T11:43:33.632 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2599999999999996e-06 2024-09-16T11:43:33.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.800472+0000", 2024-09-16T11:43:33.633 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9714999999999999e-05 2024-09-16T11:43:33.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.633 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.633 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.800490+0000", 2024-09-16T11:43:33.633 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7764000000000001e-05 2024-09-16T11:43:33.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.633 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.633 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.801377+0000", 2024-09-16T11:43:33.633 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00088644500000000003 2024-09-16T11:43:33.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.634 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.634 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.801402+0000", 2024-09-16T11:43:33.634 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5058999999999998e-05 2024-09-16T11:43:33.634 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.634 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.634 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.634 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.634 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:93 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'67, mlcod=15'64)", 2024-09-16T11:43:33.634 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:13.802440+0000", 2024-09-16T11:43:33.634 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.742505274999999, 2024-09-16T11:43:33.634 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00087350700000000004, 2024-09-16T11:43:33.634 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.635 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.635 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.635 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.635 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.802440+0000", 2024-09-16T11:43:33.635 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.635 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.635 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.802440+0000", 2024-09-16T11:43:33.635 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.636 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.636 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.802441+0000", 2024-09-16T11:43:33.636 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7199999999999999e-07 2024-09-16T11:43:33.636 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.636 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.636 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.636 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.802506+0000", 2024-09-16T11:43:33.636 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5548000000000003e-05 2024-09-16T11:43:33.636 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.636 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.636 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.636 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.802516+0000", 2024-09-16T11:43:33.637 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0366999999999999e-05 2024-09-16T11:43:33.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.637 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.637 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.637 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.802521+0000", 2024-09-16T11:43:33.637 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7520000000000004e-06 2024-09-16T11:43:33.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.637 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.637 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.637 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.802541+0000", 2024-09-16T11:43:33.637 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0088000000000001e-05 2024-09-16T11:43:33.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.637 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.802553+0000", 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1252e-05 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.803292+0000", 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00073899100000000004 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.803314+0000", 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2036999999999999e-05 2024-09-16T11:43:33.638 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.639 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.639 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.639 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.639 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4181.0:95 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'69, mlcod=15'67)", 2024-09-16T11:43:33.639 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:13.804511+0000", 2024-09-16T11:43:33.639 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.740434435000001, 2024-09-16T11:43:33.639 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00079374100000000004, 2024-09-16T11:43:33.639 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:33.639 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:33.639 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:33.639 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.639 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.804511+0000", 2024-09-16T11:43:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.804511+0000", 2024-09-16T11:43:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.804511+0000", 2024-09-16T11:43:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:43:33.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.641 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.641 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:33.641 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.804609+0000", 2024-09-16T11:43:33.641 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7649e-05 2024-09-16T11:43:33.641 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.641 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.641 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:33.641 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.804616+0000", 2024-09-16T11:43:33.641 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.17e-06 2024-09-16T11:43:33.641 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.641 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.641 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:33.642 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.804621+0000", 2024-09-16T11:43:33.642 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8450000000000002e-06 2024-09-16T11:43:33.642 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.642 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.642 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:33.642 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.804647+0000", 2024-09-16T11:43:33.642 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5485000000000001e-05 2024-09-16T11:43:33.642 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.642 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.642 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:33.642 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.804659+0000", 2024-09-16T11:43:33.642 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2629000000000001e-05 2024-09-16T11:43:33.642 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.643 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.643 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:33.643 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.805283+0000", 2024-09-16T11:43:33.643 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00062401999999999998 2024-09-16T11:43:33.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:33.643 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:33.643 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:33.643 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:13.805305+0000", 2024-09-16T11:43:33.643 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1500000000000001e-05 2024-09-16T11:43:33.643 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.643 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.643 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.643 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:33.644 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:33.644 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:33.645 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:43:33.646 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:33.643+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:33.746 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:33.744+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:33.775 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T11:43:33.790 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:33.791 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:33.791 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:33.791 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:33.801 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:43:33.846 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:33.844+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:33.942 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:33.942 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:33.942 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:33.943 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:33.943 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:33.947 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:33.946+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:33.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:43:34.047 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:34.046+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:34.050 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:34.050 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:34.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:34.050 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:34.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.050 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4247.0:5 2.2 e18/16 2:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-16T11:43:34.051 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:15.735625+0000", 2024-09-16T11:43:34.051 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.31343618, 2024-09-16T11:43:34.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00082803800000000002, 2024-09-16T11:43:34.051 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:34.051 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:34.051 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:34.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:34.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:15.735625+0000", 2024-09-16T11:43:34.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:34.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:34.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:15.735625+0000", 2024-09-16T11:43:34.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:34.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:34.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:15.735626+0000", 2024-09-16T11:43:34.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.13e-07 2024-09-16T11:43:34.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:34.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:15.735634+0000", 2024-09-16T11:43:34.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3969999999999994e-06 2024-09-16T11:43:34.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:34.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:15.735662+0000", 2024-09-16T11:43:34.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8112000000000001e-05 2024-09-16T11:43:34.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:34.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:15.735678+0000", 2024-09-16T11:43:34.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5829e-05 2024-09-16T11:43:34.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:34.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:15.735745+0000", 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6969000000000002e-05 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:15.735772+0000", 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6894e-05 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:15.736412+0000", 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00063990399999999995 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:15.736453+0000", 2024-09-16T11:43:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.142e-05 2024-09-16T11:43:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4247.0:7 2.2 e19/16 2:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-16T11:43:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:16.739021+0000", 2024-09-16T11:43:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.310040209, 2024-09-16T11:43:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00095420300000000004, 2024-09-16T11:43:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:16.739021+0000", 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:16.739021+0000", 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:16.739021+0000", 2024-09-16T11:43:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.39e-07 2024-09-16T11:43:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:16.739030+0000", 2024-09-16T11:43:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5839999999999995e-06 2024-09-16T11:43:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:16.739045+0000", 2024-09-16T11:43:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4995e-05 2024-09-16T11:43:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:16.739058+0000", 2024-09-16T11:43:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3329999999999999e-05 2024-09-16T11:43:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:16.739118+0000", 2024-09-16T11:43:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9759000000000002e-05 2024-09-16T11:43:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:16.739143+0000", 2024-09-16T11:43:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4769e-05 2024-09-16T11:43:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:16.739929+0000", 2024-09-16T11:43:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00078571800000000003 2024-09-16T11:43:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:16.739975+0000", 2024-09-16T11:43:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6709000000000003e-05 2024-09-16T11:43:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4545.0:10 4.1c e31/28 4:3831950c:::rbd_directory:head v 31'1, mlcod=0'0)", 2024-09-16T11:43:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:33.778207+0000", 2024-09-16T11:43:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.27085379999999998, 2024-09-16T11:43:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00088576599999999996, 2024-09-16T11:43:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.778207+0000", 2024-09-16T11:43:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.778207+0000", 2024-09-16T11:43:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.778208+0000", 2024-09-16T11:43:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.778213+0000", 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6840000000000001e-06 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.778221+0000", 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5979999999999999e-06 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:34.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.778227+0000", 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2870000000000001e-06 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.778318+0000", 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0276000000000001e-05 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.778336+0000", 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8498000000000001e-05 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:34.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779069+0000", 2024-09-16T11:43:34.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00073276199999999997 2024-09-16T11:43:34.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.064 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:34.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779093+0000", 2024-09-16T11:43:34.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.438e-05 2024-09-16T11:43:34.064 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.064 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:34.064 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.064 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.064 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:34.064 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:34.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:43:34.148 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:34.146+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:34.165 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:34.165 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:34.165 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:34.165 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:34.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:43:34.249 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:34.247+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:34.272 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:34.272 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:34.272 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:34.272 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:34.272 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:34.281 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:43:34.350 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:34.348+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:34.421 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4247.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-16T11:43:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:14.730801+0000", 2024-09-16T11:43:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.688829132999999, 2024-09-16T11:43:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000654092, 2024-09-16T11:43:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730801+0000", 2024-09-16T11:43:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730801+0000", 2024-09-16T11:43:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730801+0000", 2024-09-16T11:43:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9000000000000001e-07 2024-09-16T11:43:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730807+0000", 2024-09-16T11:43:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:43:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730818+0000", 2024-09-16T11:43:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0577000000000001e-05 2024-09-16T11:43:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730827+0000", 2024-09-16T11:43:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8130000000000002e-06 2024-09-16T11:43:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730864+0000", 2024-09-16T11:43:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7249000000000002e-05 2024-09-16T11:43:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730887+0000", 2024-09-16T11:43:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2492000000000001e-05 2024-09-16T11:43:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.731414+0000", 2024-09-16T11:43:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00052705899999999997 2024-09-16T11:43:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.731455+0000", 2024-09-16T11:43:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1155000000000002e-05 2024-09-16T11:43:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T11:43:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:33.777581+0000", 2024-09-16T11:43:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.64204867600000004, 2024-09-16T11:43:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001711986, 2024-09-16T11:43:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 10 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777581+0000", 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777581+0000", 2024-09-16T11:43:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777583+0000", 2024-09-16T11:43:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.584e-06 2024-09-16T11:43:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777589+0000", 2024-09-16T11:43:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6309999999999997e-06 2024-09-16T11:43:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777602+0000", 2024-09-16T11:43:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2935999999999999e-05 2024-09-16T11:43:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777612+0000", 2024-09-16T11:43:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.859e-06 2024-09-16T11:43:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777658+0000", 2024-09-16T11:43:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5380000000000003e-05 2024-09-16T11:43:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777789+0000", 2024-09-16T11:43:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013106399999999999 2024-09-16T11:43:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777992+0000", 2024-09-16T11:43:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000203764 2024-09-16T11:43:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.778848+0000", 2024-09-16T11:43:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00085536599999999998 2024-09-16T11:43:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779202+0000", 2024-09-16T11:43:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00035394000000000002 2024-09-16T11:43:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779253+0000", 2024-09-16T11:43:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1328000000000001e-05 2024-09-16T11:43:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779293+0000", 2024-09-16T11:43:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0133999999999999e-05 2024-09-16T11:43:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.4545.0:10 4.1c e31/28 ondisk, result = 0)", 2024-09-16T11:43:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:33.779134+0000", 2024-09-16T11:43:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.64049620500000004, 2024-09-16T11:43:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000169073, 2024-09-16T11:43:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779134+0000", 2024-09-16T11:43:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779134+0000", 2024-09-16T11:43:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779134+0000", 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.99e-07 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779138+0000", 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8290000000000001e-06 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779147+0000", 2024-09-16T11:43:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2070000000000001e-06 2024-09-16T11:43:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779151+0000", 2024-09-16T11:43:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9729999999999999e-06 2024-09-16T11:43:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779186+0000", 2024-09-16T11:43:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5157000000000002e-05 2024-09-16T11:43:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779195+0000", 2024-09-16T11:43:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6799999999999999e-06 2024-09-16T11:43:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779303+0000", 2024-09-16T11:43:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000107528 2024-09-16T11:43:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:34.437 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:34.438 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:43:34.450 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:34.448+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:34.526 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:43:34.537 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:34.537 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:34.546 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:43:34.587 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:34.585+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:34.641 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:43:34.642 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:43:34.651 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:34.649+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:34.658 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:43:34.658 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:43:34.673 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:43:34.674 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:43:34.689 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:43:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:43:34.690 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:34.690 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:34.699 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:43:34.706 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:43:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:43:34.722 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:43:34.722 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:43:34.738 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:43:34.738 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:43:34.752 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:34.750+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:34.754 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:43:34.754 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:43:34.806 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:34.806 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:34.815 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:43:34.852 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:34.850+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:34.923 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:34.923 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:34.932 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:43:34.954 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:34.952+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:35.030 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:35.030 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:35.039 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:43:35.083 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:35.082+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:35.136 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:35.137 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:35.145 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:43:35.155 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:35.153+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:35.166 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:43:35.167 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:43:35.167 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-16T11:43:35.167 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:43:35.167 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:43:35.167 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-16T11:43:35.256 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:35.255+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:35.266 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:35.266 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T11:43:35.266 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:35.283 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:35.283 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:35.292 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:43:35.357 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:35.355+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:35.431 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:35.431 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:35.439 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:43:35.470 INFO:teuthology.orchestra.run.smithi070.stdout:skipping zero size read 2024-09-16T11:43:35.483 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:35.482+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:35.485 INFO:teuthology.orchestra.run.smithi070.stdout:2 write 0x6e884a2 thru 0x6e89655 (0x11b4 bytes) 2024-09-16T11:43:35.488 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:35.500 INFO:teuthology.orchestra.run.smithi070.stdout:5 write 0xa8c7331 thru 0xa8c79bd (0x68d bytes) 2024-09-16T11:43:35.504 INFO:teuthology.orchestra.run.smithi070.stdout:7 read 0x92d0217 thru 0x92d7c96 (0x7a80 bytes) 2024-09-16T11:43:35.504 INFO:teuthology.orchestra.run.smithi070.stdout:8 writesame 0x6427eae thru 0x6433270 data_size 0x1(0xb3c3 bytes) 2024-09-16T11:43:35.526 INFO:teuthology.orchestra.run.smithi070.stdout:9 write 0x23f8d45 thru 0x23fba4b (0x2d07 bytes) 2024-09-16T11:43:35.529 INFO:teuthology.orchestra.run.smithi070.stdout:10 read 0x58b9cc9 thru 0x58c13d8 (0x7710 bytes) 2024-09-16T11:43:35.529 INFO:teuthology.orchestra.run.smithi070.stdout:12 punch from 0xa49c572 to 0xa4a7ff6, (0xba84 bytes) 2024-09-16T11:43:35.529 INFO:teuthology.orchestra.run.smithi070.stdout:13 write 0x394b49f thru 0x39515da (0x613c bytes) 2024-09-16T11:43:35.533 INFO:teuthology.orchestra.run.smithi070.stdout:14 write 0x9551bbd thru 0x955b33e (0x9782 bytes) 2024-09-16T11:43:35.537 INFO:teuthology.orchestra.run.smithi070.stdout:15 write 0x80a246f thru 0x80a9ea9 (0x7a3b bytes) 2024-09-16T11:43:35.540 INFO:teuthology.orchestra.run.smithi070.stdout:19 read 0x9dd0248 thru 0x9dd9d1f (0x9ad8 bytes) 2024-09-16T11:43:35.541 INFO:teuthology.orchestra.run.smithi070.stdout:20 write 0x76bff95 thru 0x76ca1f8 (0xa264 bytes) 2024-09-16T11:43:35.543 INFO:teuthology.orchestra.run.smithi070.stdout:23 read 0x80a806a thru 0x80ae1ee (0x6185 bytes) 2024-09-16T11:43:35.543 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:35.543 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:35.544 INFO:teuthology.orchestra.run.smithi070.stdout:24 punch from 0x5409f04 to 0x540ce96, (0x2f92 bytes) 2024-09-16T11:43:35.544 INFO:teuthology.orchestra.run.smithi070.stdout:25 write 0x2e1a251 thru 0x2e25022 (0xadd2 bytes) 2024-09-16T11:43:35.548 INFO:teuthology.orchestra.run.smithi070.stdout:26 writesame 0x6ff0e9f thru 0x6ff68ec data_size 0x3(0x5a4e bytes) 2024-09-16T11:43:35.552 INFO:teuthology.orchestra.run.smithi070.stdout:27 read 0x9c71a88 thru 0x9c7a5d1 (0x8b4a bytes) 2024-09-16T11:43:35.552 INFO:teuthology.orchestra.run.smithi070.stdout:28 read 0xd43c2e thru 0xd51321 (0xd6f4 bytes) 2024-09-16T11:43:35.552 INFO:teuthology.orchestra.run.smithi070.stdout:truncating to largest ever: 0xb560bd3 2024-09-16T11:43:35.553 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:43:35.554 INFO:teuthology.orchestra.run.smithi070.stdout:29 trunc from 0xa8c79be to 0xb560bd3 2024-09-16T11:43:35.564 INFO:teuthology.orchestra.run.smithi070.stdout:30 trunc from 0xb560bd3 to 0x613054b 2024-09-16T11:43:35.573 INFO:teuthology.orchestra.run.smithi070.stdout:31 write 0x47621dc thru 0x47713b9 (0xf1de bytes) 2024-09-16T11:43:35.576 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:35.580 INFO:teuthology.orchestra.run.smithi070.stdout:36 trunc from 0x613054b to 0x97e7bf9 2024-09-16T11:43:35.586 INFO:teuthology.orchestra.run.smithi070.stdout:38 read 0x25a700e thru 0x25aeea6 (0x7e99 bytes) 2024-09-16T11:43:35.594 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:35.593+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:35.596 INFO:teuthology.orchestra.run.smithi070.stdout:40 write 0xcca876d thru 0xccac38b (0x3c1f bytes) 2024-09-16T11:43:35.613 INFO:teuthology.orchestra.run.smithi070.stdout:42 trunc from 0xccac38c to 0x114ee64 2024-09-16T11:43:35.624 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:35.646 INFO:teuthology.orchestra.run.smithi070.stdout:44 write 0xe9b8865 thru 0xe9c1987 (0x9123 bytes) 2024-09-16T11:43:35.649 INFO:teuthology.orchestra.run.smithi070.stdout:truncating to largest ever: 0xc6d2e81 2024-09-16T11:43:35.652 INFO:teuthology.orchestra.run.smithi070.stdout:49 trunc from 0xe9c1988 to 0xc6d2e81 2024-09-16T11:43:35.659 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:35.657+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:35.663 INFO:teuthology.orchestra.run.smithi070.stdout:50 read 0x219b7d6 thru 0x21a747b (0xbca6 bytes) 2024-09-16T11:43:35.663 INFO:teuthology.orchestra.run.smithi070.stdout:51 punch from 0xb991808 to 0xb999ff6, (0x87ee bytes) 2024-09-16T11:43:35.664 INFO:teuthology.orchestra.run.smithi070.stdout:52 read 0x2d2faa3 thru 0x2d35176 (0x56d4 bytes) 2024-09-16T11:43:35.669 INFO:teuthology.orchestra.run.smithi070.stdout:53 trunc from 0xc6d2e81 to 0x7dbdb98 2024-09-16T11:43:35.684 INFO:teuthology.orchestra.run.smithi070.stdout:54 writesame 0xa650ad8 thru 0xa65dd8d data_size 0x7(0xd2b6 bytes) 2024-09-16T11:43:35.684 INFO:teuthology.orchestra.run.smithi070.stdout:55 read 0x73f5846 thru 0x73fa262 (0x4a1d bytes) 2024-09-16T11:43:35.684 INFO:teuthology.orchestra.run.smithi070.stdout:57 read 0x555120c thru 0x555dc53 (0xca48 bytes) 2024-09-16T11:43:35.685 INFO:teuthology.orchestra.run.smithi070.stdout:58 punch from 0xc93ad2 to 0xc9f0dd, (0xb60b bytes) 2024-09-16T11:43:35.685 INFO:teuthology.orchestra.run.smithi070.stdout:60 read 0x8027c47 thru 0x8029f59 (0x2313 bytes) 2024-09-16T11:43:35.685 INFO:teuthology.orchestra.run.smithi070.stdout:61 read 0x4468f2f thru 0x446df25 (0x4ff7 bytes) 2024-09-16T11:43:35.685 INFO:teuthology.orchestra.run.smithi070.stdout:64 writesame 0x40e05e thru 0x41cb2b data_size 0x5(0xeace bytes) 2024-09-16T11:43:35.690 INFO:teuthology.orchestra.run.smithi070.stdout:66 read 0x6be98d1 thru 0x6beb58f (0x1cbf bytes) 2024-09-16T11:43:35.690 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:35.690 INFO:teuthology.orchestra.run.smithi070.stdout:68 read 0x9ada4a thru 0x9b8af3 (0xb0aa bytes) 2024-09-16T11:43:35.690 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:35.697 INFO:teuthology.orchestra.run.smithi070.stdout:70 writesame 0xa7b2a35 thru 0xa7b3922 data_size 0x27(0xeee bytes) 2024-09-16T11:43:35.697 INFO:teuthology.orchestra.run.smithi070.stdout:71 write 0x5acca72 thru 0x5ad3924 (0x6eb3 bytes) 2024-09-16T11:43:35.700 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:35.700 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:35.703 INFO:teuthology.orchestra.run.smithi070.stdout:72 trunc from 0xa7b3923 to 0x8525354 2024-09-16T11:43:35.709 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:43:35.710 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:35.710 INFO:teuthology.orchestra.run.smithi070.stdout:76 read 0x516fc2c thru 0x517f622 (0xf9f7 bytes) 2024-09-16T11:43:35.710 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:35.710 INFO:teuthology.orchestra.run.smithi070.stdout:78 punch from 0x51c6593 to 0x51c808e, (0x1afb bytes) 2024-09-16T11:43:35.710 INFO:teuthology.orchestra.run.smithi070.stdout:79 read 0x1f05fe7 thru 0x1f146ad (0xe6c7 bytes) 2024-09-16T11:43:35.711 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:35.711 INFO:teuthology.orchestra.run.smithi070.stdout:81 write 0x740b8da thru 0x7415deb (0xa512 bytes) 2024-09-16T11:43:35.727 INFO:teuthology.orchestra.run.smithi070.stdout:82 writesame 0xeb976e6 thru 0xeb97bf0 data_size 0x1(0x50b bytes) 2024-09-16T11:43:35.727 INFO:teuthology.orchestra.run.smithi070.stdout:84 writesame 0xc0f03f9 thru 0xc0f8a00 data_size 0x4(0x8608 bytes) 2024-09-16T11:43:35.732 INFO:teuthology.orchestra.run.smithi070.stdout:85 punch from 0xce08db7 to 0xce141ab, (0xb3f4 bytes) 2024-09-16T11:43:35.732 INFO:teuthology.orchestra.run.smithi070.stdout:86 punch from 0xc23c67a to 0xc23c741, (0xc7 bytes) 2024-09-16T11:43:35.732 INFO:teuthology.orchestra.run.smithi070.stdout:truncating to largest ever: 0xd02b682 2024-09-16T11:43:35.734 INFO:teuthology.orchestra.run.smithi070.stdout:87 trunc from 0xeb97bf1 to 0xd02b682 2024-09-16T11:43:35.740 INFO:teuthology.orchestra.run.smithi070.stdout:88 writesame 0x7b6e45b thru 0x7b74b13 data_size 0x1(0x66b9 bytes) 2024-09-16T11:43:35.755 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:35.755 INFO:teuthology.orchestra.run.smithi070.stdout:91 read 0x96980a7 thru 0x96a4ddf (0xcd39 bytes) 2024-09-16T11:43:35.760 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:35.758+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:35.761 INFO:teuthology.orchestra.run.smithi070.stdout:92 trunc from 0xd02b682 to 0x82b72f9 2024-09-16T11:43:35.773 INFO:teuthology.orchestra.run.smithi070.stdout:93 read 0x5c201f4 thru 0x5c20ce3 (0xaf0 bytes) 2024-09-16T11:43:35.773 INFO:teuthology.orchestra.run.smithi070.stdout:96 write 0x1137d1f thru 0x1140f1a (0x91fc bytes) 2024-09-16T11:43:35.776 INFO:teuthology.orchestra.run.smithi070.stdout:97 write 0x1ad37be thru 0x1adec98 (0xb4db bytes) 2024-09-16T11:43:35.779 INFO:teuthology.orchestra.run.smithi070.stdout:98 write 0x1976c6f thru 0x197756d (0x8ff bytes) 2024-09-16T11:43:35.780 INFO:teuthology.orchestra.run.smithi070.stdout:99 writesame 0x367f05d thru 0x367fcab data_size 0x1(0xc4f bytes) 2024-09-16T11:43:35.791 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:43:35.795 INFO:teuthology.orchestra.run.smithi070.stdout:100 writesame 0xd93f402 thru 0xd941fc4 data_size 0x1(0x2bc3 bytes) 2024-09-16T11:43:35.795 INFO:teuthology.orchestra.run.smithi070.stdout:101 read 0x2e69de6 thru 0x2e78a88 (0xeca3 bytes) 2024-09-16T11:43:35.795 INFO:teuthology.orchestra.run.smithi070.stdout:104 read 0x1429246 thru 0x1429838 (0x5f3 bytes) 2024-09-16T11:43:35.795 INFO:teuthology.orchestra.run.smithi070.stdout:105 writesame 0x905b35e thru 0x90647d1 data_size 0x6(0x9474 bytes) 2024-09-16T11:43:35.799 INFO:teuthology.orchestra.run.smithi070.stdout:107 read 0x94e1076 thru 0x94eb5ae (0xa539 bytes) 2024-09-16T11:43:35.802 INFO:teuthology.orchestra.run.smithi070.stdout:108 trunc from 0xd941fc5 to 0xbe9e35a 2024-09-16T11:43:35.812 INFO:teuthology.orchestra.run.smithi070.stdout:109 write 0xc54e47d thru 0xc54ed07 (0x88b bytes) 2024-09-16T11:43:35.815 INFO:teuthology.orchestra.run.smithi070.stdout:110 read 0xd846bd thru 0xd94630 (0xff74 bytes) 2024-09-16T11:43:35.815 INFO:teuthology.orchestra.run.smithi070.stdout:111 write 0x796d790 thru 0x7973bc4 (0x6435 bytes) 2024-09-16T11:43:35.816 INFO:teuthology.orchestra.run.smithi070.stdout:112 write 0x33e0db9 thru 0x33e16bd (0x905 bytes) 2024-09-16T11:43:35.816 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:35.817 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:35.819 INFO:teuthology.orchestra.run.smithi070.stdout:113 write 0x1c31fe9 thru 0x1c3508e (0x30a6 bytes) 2024-09-16T11:43:35.822 INFO:teuthology.orchestra.run.smithi070.stdout:115 punch from 0xacae8b6 to 0xacbdf3e, (0xf688 bytes) 2024-09-16T11:43:35.823 INFO:teuthology.orchestra.run.smithi070.stdout:117 punch from 0x4983d24 to 0x4984c43, (0xf1f bytes) 2024-09-16T11:43:35.823 INFO:teuthology.orchestra.run.smithi070.stdout:119 punch from 0x7ebe3a4 to 0x7ecdd09, (0xf965 bytes) 2024-09-16T11:43:35.823 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:35.826 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:43:35.828 INFO:teuthology.orchestra.run.smithi070.stdout:122 trunc from 0xc54ed08 to 0x7697ca8 2024-09-16T11:43:35.852 INFO:teuthology.orchestra.run.smithi070.stdout:123 write 0xd24798f thru 0xd254f1f (0xd591 bytes) 2024-09-16T11:43:35.855 INFO:teuthology.orchestra.run.smithi070.stdout:124 punch from 0x9786802 to 0x9788108, (0x1906 bytes) 2024-09-16T11:43:35.856 INFO:teuthology.orchestra.run.smithi070.stdout:125 read 0x42d76f6 thru 0x42e4a12 (0xd31d bytes) 2024-09-16T11:43:35.856 INFO:teuthology.orchestra.run.smithi070.stdout:127 punch from 0xb56c261 to 0xb56cc4e, (0x9ed bytes) 2024-09-16T11:43:35.856 INFO:teuthology.orchestra.run.smithi070.stdout:128 write 0x3cb5dce thru 0x3cc1f86 (0xc1b9 bytes) 2024-09-16T11:43:35.859 INFO:teuthology.orchestra.run.smithi070.stdout:129 read 0x672f42a thru 0x6732d88 (0x395f bytes) 2024-09-16T11:43:35.859 INFO:teuthology.orchestra.run.smithi070.stdout:130 write 0x3f891a7 thru 0x3f933da (0xa234 bytes) 2024-09-16T11:43:35.860 INFO:teuthology.orchestra.run.smithi070.stdout:131 writesame 0x7cec20a thru 0x7cf4424 data_size 0x1(0x821b bytes) 2024-09-16T11:43:35.870 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:35.869+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:35.876 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:35.877 INFO:teuthology.orchestra.run.smithi070.stdout:133 read 0xce77e9d thru 0xce84a94 (0xcbf8 bytes) 2024-09-16T11:43:35.877 INFO:teuthology.orchestra.run.smithi070.stdout:134 punch from 0x3b0765 to 0x3b5cf3, (0x558e bytes) 2024-09-16T11:43:35.877 INFO:teuthology.orchestra.run.smithi070.stdout:136 write 0xcff7246 thru 0xd0048b1 (0xd66c bytes) 2024-09-16T11:43:35.879 INFO:teuthology.orchestra.run.smithi070.stdout:137 read 0x3925e31 thru 0x392f6a3 (0x9873 bytes) 2024-09-16T11:43:35.880 INFO:teuthology.orchestra.run.smithi070.stdout:138 write 0xafcc6cb thru 0xafd7ac7 (0xb3fd bytes) 2024-09-16T11:43:35.883 INFO:teuthology.orchestra.run.smithi070.stdout:139 write 0x5bbf3e3 thru 0x5bca4d3 (0xb0f1 bytes) 2024-09-16T11:43:35.890 INFO:teuthology.orchestra.run.smithi070.stdout:140 writesame 0xe5a6087 thru 0xe5b47f5 data_size 0x57(0xe76f bytes) 2024-09-16T11:43:35.891 INFO:teuthology.orchestra.run.smithi070.stdout:141 read 0xda0e457 thru 0xda169cc (0x8576 bytes) 2024-09-16T11:43:35.891 INFO:teuthology.orchestra.run.smithi070.stdout:142 writesame 0xde6fff6 thru 0xde7b442 data_size 0x1(0xb44d bytes) 2024-09-16T11:43:35.912 INFO:teuthology.orchestra.run.smithi070.stdout:143 read 0xb8b44f0 thru 0xb8be776 (0xa287 bytes) 2024-09-16T11:43:35.923 INFO:teuthology.orchestra.run.smithi070.stdout:144 trunc from 0xe5b47f6 to 0x4a922c3 2024-09-16T11:43:35.932 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:35.932 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:35.934 INFO:teuthology.orchestra.run.smithi070.stdout:145 writesame 0x145580c thru 0x1458747 data_size 0x2(0x2f3c bytes) 2024-09-16T11:43:35.942 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:43:35.955 INFO:teuthology.orchestra.run.smithi070.stdout:146 write 0xe4a3af6 thru 0xe4af0a8 (0xb5b3 bytes) 2024-09-16T11:43:35.958 INFO:teuthology.orchestra.run.smithi070.stdout:147 punch from 0x9258b26 to 0x925e45d, (0x5937 bytes) 2024-09-16T11:43:35.958 INFO:teuthology.orchestra.run.smithi070.stdout:149 writesame 0x4c64f5d thru 0x4c6a94d data_size 0x19(0x59f1 bytes) 2024-09-16T11:43:35.961 INFO:teuthology.orchestra.run.smithi070.stdout:150 punch from 0x35e4500 to 0x35f19b9, (0xd4b9 bytes) 2024-09-16T11:43:35.961 INFO:teuthology.orchestra.run.smithi070.stdout:151 punch from 0x15595e4 to 0x15623a5, (0x8dc1 bytes) 2024-09-16T11:43:35.961 INFO:teuthology.orchestra.run.smithi070.stdout:153 read 0x3b08627 thru 0x3b0dfd0 (0x59aa bytes) 2024-09-16T11:43:35.969 INFO:teuthology.orchestra.run.smithi070.stdout:155 trunc from 0xe4af0a9 to 0x7a24d32 2024-09-16T11:43:35.977 INFO:teuthology.orchestra.run.smithi070.stdout:156 read 0x7aa106 thru 0x7b19c9 (0x78c4 bytes) 2024-09-16T11:43:35.982 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:35.981+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:35.985 INFO:teuthology.orchestra.run.smithi070.stdout:157 write 0x99a854b thru 0x99a9dcf (0x1885 bytes) 2024-09-16T11:43:35.987 INFO:teuthology.orchestra.run.smithi070.stdout:158 punch from 0x18acd9a to 0x18b5a3f, (0x8ca5 bytes) 2024-09-16T11:43:35.999 INFO:teuthology.orchestra.run.smithi070.stdout:159 write 0xed496dc thru 0xed563fc (0xcd21 bytes) 2024-09-16T11:43:36.002 INFO:teuthology.orchestra.run.smithi070.stdout:161 read 0xc07b26 thru 0xc16539 (0xea14 bytes) 2024-09-16T11:43:36.003 INFO:teuthology.orchestra.run.smithi070.stdout:164 punch from 0x5ea3c2c to 0x5ea73ea, (0x37be bytes) 2024-09-16T11:43:36.003 INFO:teuthology.orchestra.run.smithi070.stdout:166 punch from 0xcd62590 to 0xcd68d18, (0x6788 bytes) 2024-09-16T11:43:36.003 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.003 INFO:teuthology.orchestra.run.smithi070.stdout:168 punch from 0xe919159 to 0xe91ccbc, (0x3b63 bytes) 2024-09-16T11:43:36.004 INFO:teuthology.orchestra.run.smithi070.stdout:171 read 0x3768d60 thru 0x376ba0b (0x2cac bytes) 2024-09-16T11:43:36.004 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.019 INFO:teuthology.orchestra.run.smithi070.stdout:174 trunc from 0xed563fd to 0xed3412 2024-09-16T11:43:36.044 INFO:teuthology.orchestra.run.smithi070.stdout:176 writesame 0x7ebd6cd thru 0x7ebd895 data_size 0x1(0x1c9 bytes) 2024-09-16T11:43:36.044 INFO:teuthology.orchestra.run.smithi070.stdout:177 write 0x1d8ea97 thru 0x1d9a162 (0xb6cc bytes) 2024-09-16T11:43:36.051 INFO:teuthology.orchestra.run.smithi070.stdout:178 read 0x7491b1 thru 0x751cee (0x8b3e bytes) 2024-09-16T11:43:36.051 INFO:teuthology.orchestra.run.smithi070.stdout:179 read 0x5e90d58 thru 0x5e9b806 (0xaaaf bytes) 2024-09-16T11:43:36.052 INFO:teuthology.orchestra.run.smithi070.stdout:180 punch from 0x235a305 to 0x2367556, (0xd251 bytes) 2024-09-16T11:43:36.052 INFO:teuthology.orchestra.run.smithi070.stdout:181 writesame 0x293b124 thru 0x29443f2 data_size 0x3b(0x92cf bytes) 2024-09-16T11:43:36.054 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.062 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:36.060+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:36.063 INFO:teuthology.orchestra.run.smithi070.stdout:183 write 0xc258dcd thru 0xc26444e (0xb682 bytes) 2024-09-16T11:43:36.066 INFO:teuthology.orchestra.run.smithi070.stdout:185 writesame 0x85cf9c0 thru 0x85df111 data_size 0x1(0xf752 bytes) 2024-09-16T11:43:36.086 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:36.087 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:36.096 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:43:36.104 INFO:teuthology.orchestra.run.smithi070.stdout:186 write 0xd30fdc5 thru 0xd31baf7 (0xbd33 bytes) 2024-09-16T11:43:36.107 INFO:teuthology.orchestra.run.smithi070.stdout:187 read 0x4b67e31 thru 0x4b70120 (0x82f0 bytes) 2024-09-16T11:43:36.107 INFO:teuthology.orchestra.run.smithi070.stdout:188 write 0xb9c8e2f thru 0xb9d1d32 (0x8f04 bytes) 2024-09-16T11:43:36.110 INFO:teuthology.orchestra.run.smithi070.stdout:189 write 0x98c7319 thru 0x98cd95f (0x6647 bytes) 2024-09-16T11:43:36.124 INFO:teuthology.orchestra.run.smithi070.stdout:191 trunc from 0xd31baf8 to 0x3d698f3 2024-09-16T11:43:36.134 INFO:teuthology.orchestra.run.smithi070.stdout:192 punch from 0x3a96b80 to 0x3a99573, (0x29f3 bytes) 2024-09-16T11:43:36.135 INFO:teuthology.orchestra.run.smithi070.stdout:193 read 0x46ba3 thru 0x54cb8 (0xe116 bytes) 2024-09-16T11:43:36.135 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.147 INFO:teuthology.orchestra.run.smithi070.stdout:198 write 0x899ffb2 thru 0x89a8235 (0x8284 bytes) 2024-09-16T11:43:36.157 INFO:teuthology.orchestra.run.smithi070.stdout:199 write 0xaa4f52b thru 0xaa55c93 (0x6769 bytes) 2024-09-16T11:43:36.168 INFO:teuthology.orchestra.run.smithi070.stdout:201 writesame 0xbf92900 thru 0xbf99fa8 data_size 0x1(0x76a9 bytes) 2024-09-16T11:43:36.168 INFO:teuthology.orchestra.run.smithi070.stdout:202 read 0x7b6de24 thru 0x7b780ae (0xa28b bytes) 2024-09-16T11:43:36.168 INFO:teuthology.orchestra.run.smithi070.stdout:203 write 0x933c837 thru 0x933edf0 (0x25ba bytes) 2024-09-16T11:43:36.175 INFO:teuthology.orchestra.run.smithi070.stdout:205 trunc from 0xbf99fa9 to 0x8d3f4f6 2024-09-16T11:43:36.180 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:36.179+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:36.189 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.189 INFO:teuthology.orchestra.run.smithi070.stdout:207 trunc from 0x8d3f4f6 to 0x8f27128 2024-09-16T11:43:36.199 INFO:teuthology.orchestra.run.smithi070.stdout:208 trunc from 0x8f27128 to 0x463a693 2024-09-16T11:43:36.210 INFO:teuthology.orchestra.run.smithi070.stdout:209 punch from 0x9b340c to 0x9b81c0, (0x4db4 bytes) 2024-09-16T11:43:36.211 INFO:teuthology.orchestra.run.smithi070.stdout:210 trunc from 0x463a693 to 0x35d27c1 2024-09-16T11:43:36.218 INFO:teuthology.orchestra.run.smithi070.stdout:211 read 0x90c5a7 thru 0x91ae5d (0xe8b7 bytes) 2024-09-16T11:43:36.218 INFO:teuthology.orchestra.run.smithi070.stdout:213 punch from 0x182227f to 0x182ffa0, (0xdd21 bytes) 2024-09-16T11:43:36.218 INFO:teuthology.orchestra.run.smithi070.stdout:214 read 0x2b9cfc0 thru 0x2ba15c2 (0x4603 bytes) 2024-09-16T11:43:36.218 INFO:teuthology.orchestra.run.smithi070.stdout:215 read 0xb43369 thru 0xb52790 (0xf428 bytes) 2024-09-16T11:43:36.236 INFO:teuthology.orchestra.run.smithi070.stdout:217 write 0xce62908 thru 0xce7281f (0xff18 bytes) 2024-09-16T11:43:36.239 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.239 INFO:teuthology.orchestra.run.smithi070.stdout:220 write 0x85daf97 thru 0x85e5556 (0xa5c0 bytes) 2024-09-16T11:43:36.243 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:36.243 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:36.249 INFO:teuthology.orchestra.run.smithi070.stdout:226 write 0xd847054 thru 0xd8510f0 (0xa09d bytes) 2024-09-16T11:43:36.252 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:43:36.254 INFO:teuthology.orchestra.run.smithi070.stdout:227 write 0x248cc8 thru 0x256dfd (0xe136 bytes) 2024-09-16T11:43:36.257 INFO:teuthology.orchestra.run.smithi070.stdout:228 write 0xa7d91c1 thru 0xa7e74a1 (0xe2e1 bytes) 2024-09-16T11:43:36.261 INFO:teuthology.orchestra.run.smithi070.stdout:229 writesame 0x5bf8854 thru 0x5c03707 data_size 0x6(0xaeb4 bytes) 2024-09-16T11:43:36.265 INFO:teuthology.orchestra.run.smithi070.stdout:230 read 0x656e1 thru 0x6bf4c (0x686c bytes) 2024-09-16T11:43:36.266 INFO:teuthology.orchestra.run.smithi070.stdout:231 read 0x542ebf3 thru 0x543c5f9 (0xda07 bytes) 2024-09-16T11:43:36.266 INFO:teuthology.orchestra.run.smithi070.stdout:232 writesame 0x87cfc27 thru 0x87df060 data_size 0x2b(0xf43a bytes) 2024-09-16T11:43:36.267 INFO:teuthology.orchestra.run.smithi070.stdout:233 writesame 0x4b8763 thru 0x4c4f1e data_size 0x6(0xc7bc bytes) 2024-09-16T11:43:36.272 INFO:teuthology.orchestra.run.smithi070.stdout:234 punch from 0x274cb6f to 0x275ca39, (0xfeca bytes) 2024-09-16T11:43:36.272 INFO:teuthology.orchestra.run.smithi070.stdout:235 read 0x38031df thru 0x3803731 (0x553 bytes) 2024-09-16T11:43:36.272 INFO:teuthology.orchestra.run.smithi070.stdout:237 punch from 0xc41de55 to 0xc42760f, (0x97ba bytes) 2024-09-16T11:43:36.272 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.272 INFO:teuthology.orchestra.run.smithi070.stdout:240 write 0x22a8811 thru 0x22a8c58 (0x448 bytes) 2024-09-16T11:43:36.281 INFO:teuthology.orchestra.run.smithi070.stdout:244 write 0xe341f08 thru 0xe344f4a (0x3043 bytes) 2024-09-16T11:43:36.284 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.284 INFO:teuthology.orchestra.run.smithi070.stdout:247 read 0xcee78a1 thru 0xcee983d (0x1f9d bytes) 2024-09-16T11:43:36.285 INFO:teuthology.orchestra.run.smithi070.stdout:248 write 0xa00eee0 thru 0xa010ac8 (0x1be9 bytes) 2024-09-16T11:43:36.288 INFO:teuthology.orchestra.run.smithi070.stdout:249 punch from 0x2937eff to 0x293b3f7, (0x34f8 bytes) 2024-09-16T11:43:36.288 INFO:teuthology.orchestra.run.smithi070.stdout:251 read 0x34a961c thru 0x34aef9e (0x5983 bytes) 2024-09-16T11:43:36.288 INFO:teuthology.orchestra.run.smithi070.stdout:252 writesame 0x3ce7dc9 thru 0x3cf3014 data_size 0x2(0xb24c bytes) 2024-09-16T11:43:36.297 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:36.292+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:36.300 INFO:teuthology.orchestra.run.smithi070.stdout:253 writesame 0x4ee2b08 thru 0x4ef00e0 data_size 0x1(0xd5d9 bytes) 2024-09-16T11:43:36.327 INFO:teuthology.orchestra.run.smithi070.stdout:254 write 0xdcb17f2 thru 0xdcbd41c (0xbc2b bytes) 2024-09-16T11:43:36.330 INFO:teuthology.orchestra.run.smithi070.stdout:255 read 0x22f4cbc thru 0x22f8475 (0x37ba bytes) 2024-09-16T11:43:36.330 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.330 INFO:teuthology.orchestra.run.smithi070.stdout:257 read 0x4c756b8 thru 0x4c78a8d (0x33d6 bytes) 2024-09-16T11:43:36.331 INFO:teuthology.orchestra.run.smithi070.stdout:258 write 0x6b3f396 thru 0x6b3fff5 (0xc60 bytes) 2024-09-16T11:43:36.334 INFO:teuthology.orchestra.run.smithi070.stdout:259 writesame 0x4ba4aef thru 0x4bb2b4a data_size 0xa6(0xe05c bytes) 2024-09-16T11:43:36.344 INFO:teuthology.orchestra.run.smithi070.stdout:261 trunc from 0xe344f4b to 0x905685c 2024-09-16T11:43:36.359 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.359 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.359 INFO:teuthology.orchestra.run.smithi070.stdout:265 read 0x3a15d5e thru 0x3a1e648 (0x88eb bytes) 2024-09-16T11:43:36.359 INFO:teuthology.orchestra.run.smithi070.stdout:269 read 0x6ba210a thru 0x6baed13 (0xcc0a bytes) 2024-09-16T11:43:36.364 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:36.363+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:36.368 INFO:teuthology.orchestra.run.smithi070.stdout:270 write 0xae4e916 thru 0xae54637 (0x5d22 bytes) 2024-09-16T11:43:36.371 INFO:teuthology.orchestra.run.smithi070.stdout:271 punch from 0x13e04ac to 0x13e7400, (0x6f54 bytes) 2024-09-16T11:43:36.371 INFO:teuthology.orchestra.run.smithi070.stdout:272 read 0xa9c6bb7 thru 0xa9ca751 (0x3b9b bytes) 2024-09-16T11:43:36.371 INFO:teuthology.orchestra.run.smithi070.stdout:275 clone 1 order 22 su 262144 sc 4 2024-09-16T11:43:36.399 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:36.399 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:36.408 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:43:36.466 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:36.464+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:36.506 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:36.506 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:36.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:43:36.566 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:36.565+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:36.611 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:36.612 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:36.620 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:43:36.667 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:36.665+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:36.719 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:36.719 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:36.728 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:43:36.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:43:36.772 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:36.770+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:36.784 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T11:43:36.832 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:36.833 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:36.842 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:43:36.868 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:36.866+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:36.896 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:43:36.897 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:43:36.913 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:43:36.913 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:43:36.928 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:43:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:43:36.943 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:43:36.944 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:43:36.956 INFO:teuthology.orchestra.run.smithi070.stdout:276 write 0xbec923b thru 0xbed59eb (0xc7b1 bytes) 2024-09-16T11:43:36.960 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.960 INFO:teuthology.orchestra.run.smithi070.stdout:278 read 0x6dc3412 thru 0x6dcbae8 (0x86d7 bytes) 2024-09-16T11:43:36.960 INFO:teuthology.orchestra.run.smithi070.stdout:280 read 0x84134f0 thru 0x8419f15 (0x6a26 bytes) 2024-09-16T11:43:36.960 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.960 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:36.960 INFO:teuthology.orchestra.run.smithi070.stdout:283 read 0x4d40016 thru 0x4d41024 (0x100f bytes) 2024-09-16T11:43:36.961 INFO:teuthology.orchestra.run.smithi070.stdout:284 punch from 0x6ec6e9a to 0x6ecb108, (0x426e bytes) 2024-09-16T11:43:36.961 INFO:teuthology.orchestra.run.smithi070.stdout:285 trunc from 0xbed59ec to 0xb80df07 2024-09-16T11:43:36.967 INFO:teuthology.orchestra.run.smithi070.stdout:287 trunc from 0xb80df07 to 0xc146f80 2024-09-16T11:43:36.969 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:36.967+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:36.972 INFO:teuthology.orchestra.run.smithi070.stdout:288 write 0x25d84bd thru 0x25e60ac (0xdbf0 bytes) 2024-09-16T11:43:36.972 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:43:36.973 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:43:36.976 INFO:teuthology.orchestra.run.smithi070.stdout:289 read 0xae7ca76 thru 0xae7fa0e (0x2f99 bytes) 2024-09-16T11:43:36.976 INFO:teuthology.orchestra.run.smithi070.stdout:291 write 0x767a5e4 thru 0x768792f (0xd34c bytes) 2024-09-16T11:43:36.980 INFO:teuthology.orchestra.run.smithi070.stdout:292 punch from 0x6a0d27b to 0x6a1c776, (0xf4fb bytes) 2024-09-16T11:43:36.980 INFO:teuthology.orchestra.run.smithi070.stdout:293 writesame 0x5c4f084 thru 0x5c5017c data_size 0xb(0x10f9 bytes) 2024-09-16T11:43:36.984 INFO:teuthology.orchestra.run.smithi070.stdout:294 read 0x68b160c thru 0x68b24f4 (0xee9 bytes) 2024-09-16T11:43:36.984 INFO:teuthology.orchestra.run.smithi070.stdout:297 write 0x7ea554c thru 0x7ea9ca6 (0x475b bytes) 2024-09-16T11:43:36.990 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:43:36.990 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:43:36.997 INFO:teuthology.orchestra.run.smithi070.stdout:299 trunc from 0xc146f80 to 0x1ab33e6 2024-09-16T11:43:36.998 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:36.998 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:36.999 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:36.999 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:36.999 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:37.008 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:43:37.008 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:43:37.009 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:43:37.024 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:43:37.024 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:43:37.043 INFO:teuthology.orchestra.run.smithi070.stdout:300 writesame 0xcfe675d thru 0xcfeb1f9 data_size 0x1(0x4a9d bytes) 2024-09-16T11:43:37.043 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:37.043 INFO:teuthology.orchestra.run.smithi070.stdout:302 read 0x66504cf thru 0x66551c8 (0x4cfa bytes) 2024-09-16T11:43:37.044 INFO:teuthology.orchestra.run.smithi070.stdout:303 write 0xaed431d thru 0xaed6d58 (0x2a3c bytes) 2024-09-16T11:43:37.046 INFO:teuthology.orchestra.run.smithi070.stdout:304 read 0x7d559d0 thru 0x7d5a40d (0x4a3e bytes) 2024-09-16T11:43:37.046 INFO:teuthology.orchestra.run.smithi070.stdout:305 writesame 0xce5343e thru 0xce61b03 data_size 0x6d(0xe6c6 bytes) 2024-09-16T11:43:37.049 INFO:teuthology.orchestra.run.smithi070.stdout:306 write 0x6a548ea thru 0x6a54d8a (0x4a1 bytes) 2024-09-16T11:43:37.051 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:37.051 INFO:teuthology.orchestra.run.smithi070.stdout:310 read 0x1659a9e thru 0x1664bf8 (0xb15b bytes) 2024-09-16T11:43:37.052 INFO:teuthology.orchestra.run.smithi070.stdout:311 writesame 0x3c3492b thru 0x3c4189e data_size 0x22(0xcf74 bytes) 2024-09-16T11:43:37.059 INFO:teuthology.orchestra.run.smithi070.stdout:312 write 0x89bb2a9 thru 0x89c7488 (0xc1e0 bytes) 2024-09-16T11:43:37.066 INFO:teuthology.orchestra.run.smithi070.stdout:313 read 0xc55b106 thru 0xc563c35 (0x8b30 bytes) 2024-09-16T11:43:37.067 INFO:teuthology.orchestra.run.smithi070.stdout:314 punch from 0xc9b5564 to 0xc9c400c, (0xeaa8 bytes) 2024-09-16T11:43:37.067 INFO:teuthology.orchestra.run.smithi070.stdout:315 read 0x295a77a thru 0x295b525 (0xdac bytes) 2024-09-16T11:43:37.067 INFO:teuthology.orchestra.run.smithi070.stdout:316 write 0x6d58926 thru 0x6d65d03 (0xd3de bytes) 2024-09-16T11:43:37.069 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:37.068+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:37.070 INFO:teuthology.orchestra.run.smithi070.stdout:318 read 0x6f7f24c thru 0x6f86491 (0x7246 bytes) 2024-09-16T11:43:37.070 INFO:teuthology.orchestra.run.smithi070.stdout:319 read 0xc1766f0 thru 0xc17e0af (0x79c0 bytes) 2024-09-16T11:43:37.070 INFO:teuthology.orchestra.run.smithi070.stdout:321 writesame 0x2306e08 thru 0x231028e data_size 0x1(0x9487 bytes) 2024-09-16T11:43:37.089 INFO:teuthology.orchestra.run.smithi070.stdout:truncating to largest ever: 0xdfbee11 2024-09-16T11:43:37.090 INFO:teuthology.orchestra.run.smithi070.stdout:322 trunc from 0xcfeb1fa to 0xdfbee11 2024-09-16T11:43:37.097 INFO:teuthology.orchestra.run.smithi070.stdout:323 read 0x4e710c5 thru 0x4e79ccd (0x8c09 bytes) 2024-09-16T11:43:37.097 INFO:teuthology.orchestra.run.smithi070.stdout:324 read 0x91c20b6 thru 0x91c240b (0x356 bytes) 2024-09-16T11:43:37.105 INFO:teuthology.orchestra.run.smithi070.stdout:326 write 0xe5182df thru 0xe51dbfb (0x591d bytes) 2024-09-16T11:43:37.109 INFO:teuthology.orchestra.run.smithi070.stdout:327 read 0x55b0fae thru 0x55ba2b0 (0x9303 bytes) 2024-09-16T11:43:37.109 INFO:teuthology.orchestra.run.smithi070.stdout:328 read 0x2377d9e thru 0x23876d3 (0xf936 bytes) 2024-09-16T11:43:37.109 INFO:teuthology.orchestra.run.smithi070.stdout:329 writesame 0x2c7db11 thru 0x2c80d08 data_size 0x52(0x31f8 bytes) 2024-09-16T11:43:37.117 INFO:teuthology.orchestra.run.smithi070.stdout:330 read 0x9fe39d thru 0x9feab7 (0x71b bytes) 2024-09-16T11:43:37.117 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:37.118 INFO:teuthology.orchestra.run.smithi070.stdout:332 punch from 0x53188d8 to 0x5325913, (0xd03b bytes) 2024-09-16T11:43:37.118 INFO:teuthology.orchestra.run.smithi070.stdout:333 read 0x1daa4aa thru 0x1db29d4 (0x852b bytes) 2024-09-16T11:43:37.118 INFO:teuthology.orchestra.run.smithi070.stdout:334 read 0x2571b7f thru 0x257f7d6 (0xdc58 bytes) 2024-09-16T11:43:37.118 INFO:teuthology.orchestra.run.smithi070.stdout:335 punch from 0x9c6aa52 to 0x9c72119, (0x76c7 bytes) 2024-09-16T11:43:37.118 INFO:teuthology.orchestra.run.smithi070.stdout:336 trunc from 0xe51dbfc to 0xc8909b2 2024-09-16T11:43:37.125 INFO:teuthology.orchestra.run.smithi070.stdout:337 writesame 0xb81b85 thru 0xb85775 data_size 0x37(0x3bf1 bytes) 2024-09-16T11:43:37.129 INFO:teuthology.orchestra.run.smithi070.stdout:339 read 0x588fe4c thru 0x58951cc (0x5381 bytes) 2024-09-16T11:43:37.129 INFO:teuthology.orchestra.run.smithi070.stdout:340 write 0x2479b9f thru 0x2487331 (0xd793 bytes) 2024-09-16T11:43:37.131 INFO:teuthology.orchestra.run.smithi070.stdout:341 punch from 0xd3c388 to 0xd495f2, (0xd26a bytes) 2024-09-16T11:43:37.135 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:788 4.f 4:f25e2b08:::rbd_data.11c17b8dc4e5.000000000000002f:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3707451~51121] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:36.957171+0000", 2024-09-16T11:43:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.173372426, 2024-09-16T11:43:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001617942, 2024-09-16T11:43:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 788 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.957171+0000", 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.957171+0000", 2024-09-16T11:43:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.957173+0000", 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0219999999999999e-06 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.957215+0000", 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2314000000000002e-05 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.957235+0000", 2024-09-16T11:43:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9513000000000001e-05 2024-09-16T11:43:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.957242+0000", 2024-09-16T11:43:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4889999999999998e-06 2024-09-16T11:43:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.957298+0000", 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.558e-05 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.957450+0000", 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000152769 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3", 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.957612+0000", 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016129 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.958614+0000", 2024-09-16T11:43:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010023429999999999 2024-09-16T11:43:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.958718+0000", 2024-09-16T11:43:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103531 2024-09-16T11:43:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.958748+0000", 2024-09-16T11:43:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0202999999999999e-05 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.958789+0000", 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0887999999999999e-05 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:791 4.f 4:f25e2b08:::rbd_data.11c17b8dc4e5.000000000000002f:head [truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:36.960263+0000", 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.170280232, 2024-09-16T11:43:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001041211, 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 791 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.960263+0000", 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.960263+0000", 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.960265+0000", 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.657e-06 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.960272+0000", 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5070000000000003e-06 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.960289+0000", 2024-09-16T11:43:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6918999999999998e-05 2024-09-16T11:43:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.960295+0000", 2024-09-16T11:43:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9020000000000004e-06 2024-09-16T11:43:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.960335+0000", 2024-09-16T11:43:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9888999999999997e-05 2024-09-16T11:43:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.960395+0000", 2024-09-16T11:43:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9703000000000001e-05 2024-09-16T11:43:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3", 2024-09-16T11:43:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.960485+0000", 2024-09-16T11:43:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0593999999999994e-05 2024-09-16T11:43:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.961099+0000", 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00061351400000000001 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.961252+0000", 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015324699999999999 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.961281+0000", 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8756999999999999e-05 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.961304+0000", 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3521999999999999e-05 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:819 4.12 4:4b7766cc:::rbd_data.11c17b8dc4e5.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1410237~56304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:36.973761+0000", 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.156781948, 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001729097, 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 819 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.973761+0000", 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.973761+0000", 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.973763+0000", 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1940000000000001e-06 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.973807+0000", 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3690999999999999e-05 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.973828+0000", 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0543999999999998e-05 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.973835+0000", 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6809999999999998e-06 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.973890+0000", 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5090000000000003e-05 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.974043+0000", 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000152094 2024-09-16T11:43:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.974269+0000", 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022690399999999999 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.975296+0000", 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001026363 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.975399+0000", 2024-09-16T11:43:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103528 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.975450+0000", 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0973999999999997e-05 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.975490+0000", 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0034000000000003e-05 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:822 4.2 4:4447a4c2:::rbd_data.11c17b8dc4e5.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1811940~23068] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:36.977383+0000", 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.15316066, 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015872550000000001, 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 822 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977383+0000", 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977383+0000", 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977383+0000", 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1200000000000001e-07 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977394+0000", 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1051e-05 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977405+0000", 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.076e-05 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977410+0000", 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.408e-06 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977456+0000", 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6332999999999998e-05 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977612+0000", 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015561000000000001 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977848+0000", 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00023593400000000001 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.978737+0000", 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00088966800000000003 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.978905+0000", 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016770399999999999 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.978935+0000", 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.993e-05 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.978970+0000", 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4944999999999998e-05 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:837 4.12 4:4b7766cc:::rbd_data.11c17b8dc4e5.000000000000000b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:36.998916+0000", 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.13162684799999999, 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001424622, 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 837 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.998916+0000", 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.998916+0000", 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.998919+0000", 2024-09-16T11:43:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2050000000000001e-06 2024-09-16T11:43:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.998929+0000", 2024-09-16T11:43:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0295000000000001e-05 2024-09-16T11:43:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.998948+0000", 2024-09-16T11:43:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8828999999999999e-05 2024-09-16T11:43:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.998955+0000", 2024-09-16T11:43:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7810000000000005e-06 2024-09-16T11:43:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999012+0000", 2024-09-16T11:43:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6348000000000003e-05 2024-09-16T11:43:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999130+0000", 2024-09-16T11:43:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000118476 2024-09-16T11:43:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999251+0000", 2024-09-16T11:43:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012082999999999999 2024-09-16T11:43:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.000205+0000", 2024-09-16T11:43:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00095432199999999996 2024-09-16T11:43:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.000267+0000", 2024-09-16T11:43:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1841000000000006e-05 2024-09-16T11:43:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.000306+0000", 2024-09-16T11:43:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8426000000000001e-05 2024-09-16T11:43:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.000341+0000", 2024-09-16T11:43:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5268999999999997e-05 2024-09-16T11:43:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:839 4.2 4:4447a4c2:::rbd_data.11c17b8dc4e5.000000000000001d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:36.998977+0000", 2024-09-16T11:43:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.13156631599999999, 2024-09-16T11:43:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011952429999999999, 2024-09-16T11:43:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 839 2024-09-16T11:43:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.998977+0000", 2024-09-16T11:43:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.998977+0000", 2024-09-16T11:43:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.998977+0000", 2024-09-16T11:43:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:43:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.998980+0000", 2024-09-16T11:43:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4260000000000002e-06 2024-09-16T11:43:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.998984+0000", 2024-09-16T11:43:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3329999999999998e-06 2024-09-16T11:43:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.998991+0000", 2024-09-16T11:43:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6869999999999997e-06 2024-09-16T11:43:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999045+0000", 2024-09-16T11:43:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4280999999999997e-05 2024-09-16T11:43:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999128+0000", 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2810000000000002e-05 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999251+0000", 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000123492 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999983+0000", 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00073205099999999995 2024-09-16T11:43:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.000084+0000", 2024-09-16T11:43:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000101061 2024-09-16T11:43:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.000136+0000", 2024-09-16T11:43:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1301999999999998e-05 2024-09-16T11:43:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.000172+0000", 2024-09-16T11:43:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.642e-05 2024-09-16T11:43:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:841 4.f 4:f25e2b08:::rbd_data.11c17b8dc4e5.000000000000002f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:36.999013+0000", 2024-09-16T11:43:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.13153009800000001, 2024-09-16T11:43:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015616549999999999, 2024-09-16T11:43:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 841 2024-09-16T11:43:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999013+0000", 2024-09-16T11:43:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999013+0000", 2024-09-16T11:43:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999014+0000", 2024-09-16T11:43:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:43:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999016+0000", 2024-09-16T11:43:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4540000000000001e-06 2024-09-16T11:43:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999023+0000", 2024-09-16T11:43:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.165e-06 2024-09-16T11:43:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999028+0000", 2024-09-16T11:43:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4379999999999997e-06 2024-09-16T11:43:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999071+0000", 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3158999999999998e-05 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999150+0000", 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8858999999999998e-05 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3", 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999251+0000", 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010153300000000001 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.000249+0000", 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00099759099999999993 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.000502+0000", 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00025310699999999997 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.000543+0000", 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0707000000000002e-05 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.000575+0000", 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2178000000000003e-05 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:849 4.f 4:f7ca8ac1:::rbd_data.11c1e24d7fee.0000000000000007:4 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.004562+0000", 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.12598119099999999, 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001094947, 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 849 2024-09-16T11:43:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004562+0000", 2024-09-16T11:43:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004562+0000", 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004562+0000", 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004564+0000", 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.641e-06 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004570+0000", 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8490000000000002e-06 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004573+0000", 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4340000000000001e-06 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.005553+0000", 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00097987000000000009 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.005597+0000", 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3513999999999997e-05 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.005657+0000", 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0282999999999999e-05 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:850 4.f 4:f7ca8ac1:::rbd_data.11c1e24d7fee.0000000000000007:4 [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.004577+0000", 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.125966092, 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001246432, 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 850 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004577+0000", 2024-09-16T11:43:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004577+0000", 2024-09-16T11:43:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004577+0000", 2024-09-16T11:43:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3599999999999999e-07 2024-09-16T11:43:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004579+0000", 2024-09-16T11:43:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6390000000000001e-06 2024-09-16T11:43:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004585+0000", 2024-09-16T11:43:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5289999999999999e-06 2024-09-16T11:43:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004589+0000", 2024-09-16T11:43:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.005686+0000", 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010970109999999999 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.005755+0000", 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8507e-05 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.005824+0000", 2024-09-16T11:43:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8590999999999994e-05 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:851 4.f 4:f7ca8ac1:::rbd_data.11c1e24d7fee.0000000000000007:4 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.004594+0000", 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.125949276, 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0013588960000000001, 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 851 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004594+0000", 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004594+0000", 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004594+0000", 2024-09-16T11:43:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:43:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004596+0000", 2024-09-16T11:43:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T11:43:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004601+0000", 2024-09-16T11:43:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4079999999999997e-06 2024-09-16T11:43:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004610+0000", 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7620000000000001e-06 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.005852+0000", 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012418970000000001 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.005891+0000", 2024-09-16T11:43:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8671000000000003e-05 2024-09-16T11:43:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.005953+0000", 2024-09-16T11:43:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2148000000000002e-05 2024-09-16T11:43:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:852 4.f 4:f7ca8ac1:::rbd_data.11c1e24d7fee.0000000000000007:4 [sparse-read 1572864~46796] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.004616+0000", 2024-09-16T11:43:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.12592745499999999, 2024-09-16T11:43:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001684994, 2024-09-16T11:43:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 852 2024-09-16T11:43:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004616+0000", 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004616+0000", 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004616+0000", 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6699999999999999e-07 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004622+0000", 2024-09-16T11:43:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8309999999999997e-06 2024-09-16T11:43:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004627+0000", 2024-09-16T11:43:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4909999999999996e-06 2024-09-16T11:43:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004631+0000", 2024-09-16T11:43:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2610000000000002e-06 2024-09-16T11:43:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.005980+0000", 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0013493419999999999 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.006021+0000", 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1223999999999997e-05 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.006301+0000", 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000279478 2024-09-16T11:43:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:853 4.f 4:f7ca8ac1:::rbd_data.11c1e24d7fee.0000000000000007:4 [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.004634+0000", 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.12590880600000001, 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018127810000000001, 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 853 2024-09-16T11:43:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004634+0000", 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004634+0000", 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004636+0000", 2024-09-16T11:43:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.159e-06 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004637+0000", 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6610000000000001e-06 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004643+0000", 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5169999999999999e-06 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004646+0000", 2024-09-16T11:43:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2729999999999998e-06 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.006317+0000", 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001670971 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.006382+0000", 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5372999999999999e-05 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.006447+0000", 2024-09-16T11:43:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4826999999999994e-05 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:854 4.f 4:f7ca8ac1:::rbd_data.11c1e24d7fee.0000000000000007:4 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.004663+0000", 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.12588055000000001, 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020102369999999998, 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 854 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004663+0000", 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004663+0000", 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004663+0000", 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.06e-07 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004664+0000", 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3030000000000001e-06 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004669+0000", 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9420000000000003e-06 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004673+0000", 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.006483+0000", 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018106520000000001 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.006557+0000", 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3660999999999998e-05 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.006673+0000", 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000115764 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:855 4.f 4:f7ca8ac1:::rbd_data.11c1e24d7fee.0000000000000007:4 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.004680+0000", 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.125862797, 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002137959, 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 855 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004680+0000", 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004680+0000", 2024-09-16T11:43:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004681+0000", 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.89e-07 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004683+0000", 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0109999999999999e-06 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004689+0000", 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9869999999999996e-06 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004692+0000", 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4920000000000002e-06 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.006695+0000", 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020022529999999998 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.006744+0000", 2024-09-16T11:43:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9778999999999999e-05 2024-09-16T11:43:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.006818+0000", 2024-09-16T11:43:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3948000000000004e-05 2024-09-16T11:43:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:856 4.f 4:f7ca8ac1:::rbd_data.11c1e24d7fee.0000000000000007:4 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.004696+0000", 2024-09-16T11:43:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.12584717600000001, 2024-09-16T11:43:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0023192249999999998, 2024-09-16T11:43:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 856 2024-09-16T11:43:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004696+0000", 2024-09-16T11:43:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004696+0000", 2024-09-16T11:43:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004696+0000", 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9700000000000003e-07 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004698+0000", 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8780000000000001e-06 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004704+0000", 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4659999999999998e-06 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004707+0000", 2024-09-16T11:43:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1269999999999999e-06 2024-09-16T11:43:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.006851+0000", 2024-09-16T11:43:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021446109999999998 2024-09-16T11:43:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.006925+0000", 2024-09-16T11:43:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3522999999999994e-05 2024-09-16T11:43:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007015+0000", 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0322999999999996e-05 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:857 4.f 4:f7ca8ac1:::rbd_data.11c1e24d7fee.0000000000000007:4 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.004710+0000", 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.12583315, 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00250958, 2024-09-16T11:43:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 857 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004710+0000", 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004710+0000", 2024-09-16T11:43:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004710+0000", 2024-09-16T11:43:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2899999999999999e-07 2024-09-16T11:43:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004712+0000", 2024-09-16T11:43:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.671e-06 2024-09-16T11:43:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004718+0000", 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1e-06 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004721+0000", 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.18e-06 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007057+0000", 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0023360870000000001 2024-09-16T11:43:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007125+0000", 2024-09-16T11:43:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7793999999999995e-05 2024-09-16T11:43:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007220+0000", 2024-09-16T11:43:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4419000000000005e-05 2024-09-16T11:43:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:858 4.f 4:f7ca8ac1:::rbd_data.11c1e24d7fee.0000000000000007:4 [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.004725+0000", 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.12581822600000001, 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0026535410000000001, 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 858 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004725+0000", 2024-09-16T11:43:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004725+0000", 2024-09-16T11:43:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004725+0000", 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4499999999999998e-07 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004732+0000", 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6719999999999998e-06 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004738+0000", 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.891e-06 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.004741+0000", 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2780000000000002e-06 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007252+0000", 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025108330000000001 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007316+0000", 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3895999999999999e-05 2024-09-16T11:43:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007378+0000", 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2626000000000007e-05 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:862 4.7 4:e7dc364e:::rbd_data.11c17b8dc4e5.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.007055+0000", 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.123487706, 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0023164180000000002, 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 862 2024-09-16T11:43:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007055+0000", 2024-09-16T11:43:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007055+0000", 2024-09-16T11:43:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007057+0000", 2024-09-16T11:43:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4309999999999999e-06 2024-09-16T11:43:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007062+0000", 2024-09-16T11:43:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5489999999999997e-06 2024-09-16T11:43:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007075+0000", 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2303000000000001e-05 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007081+0000", 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1299999999999998e-06 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007247+0000", 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016606300000000001 2024-09-16T11:43:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007384+0000", 2024-09-16T11:43:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013711800000000001 2024-09-16T11:43:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.007587+0000", 2024-09-16T11:43:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000202661 2024-09-16T11:43:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.008187+0000", 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00060051799999999997 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.009323+0000", 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011354570000000001 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.009353+0000", 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9844000000000001e-05 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.009372+0000", 2024-09-16T11:43:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9344e-05 2024-09-16T11:43:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:898 4.2 4:401fb6a0:::rbd_data.11c17b8dc4e5.0000000000000022:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2601641~19799] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.061049+0000", 2024-09-16T11:43:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.069494457999999995, 2024-09-16T11:43:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0026342980000000002, 2024-09-16T11:43:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 898 2024-09-16T11:43:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.061049+0000", 2024-09-16T11:43:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.061049+0000", 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.061050+0000", 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4920000000000001e-06 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.061064+0000", 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3721e-05 2024-09-16T11:43:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.061078+0000", 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4464000000000001e-05 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.061084+0000", 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.13e-06 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.061141+0000", 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7488000000000003e-05 2024-09-16T11:43:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.061282+0000", 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014058300000000001 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.061445+0000", 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016366500000000001 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.062360+0000", 2024-09-16T11:43:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00091451300000000004 2024-09-16T11:43:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.063601+0000", 2024-09-16T11:43:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012413560000000001 2024-09-16T11:43:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.063647+0000", 2024-09-16T11:43:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5643000000000003e-05 2024-09-16T11:43:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.063683+0000", 2024-09-16T11:43:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6242999999999998e-05 2024-09-16T11:43:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:901 4.7 4:e4d3be82:::rbd_data.11c17b8dc4e5.0000000000000019:head [write 3508518~54238] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.066710+0000", 2024-09-16T11:43:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.063833124000000005, 2024-09-16T11:43:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002288871, 2024-09-16T11:43:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 901 2024-09-16T11:43:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.066710+0000", 2024-09-16T11:43:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.066710+0000", 2024-09-16T11:43:37.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.066711+0000", 2024-09-16T11:43:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8999999999999995e-07 2024-09-16T11:43:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.066734+0000", 2024-09-16T11:43:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3346999999999999e-05 2024-09-16T11:43:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.066745+0000", 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1105e-05 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.066749+0000", 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0759999999999996e-06 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.066791+0000", 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1539999999999999e-05 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.066846+0000", 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4558000000000002e-05 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.066927+0000", 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1235000000000004e-05 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.067595+0000", 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066792700000000002 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.068904+0000", 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0013094039999999999 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.068957+0000", 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2463000000000002e-05 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.068999+0000", 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2326000000000001e-05 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.240 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.240 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.240 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:37.240 INFO:teuthology.orchestra.run.smithi070.stdout:343 trunc from 0xc8909b2 to 0x48c8cf5 2024-09-16T11:43:37.241 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:43:37.243 INFO:teuthology.orchestra.run.smithi070.stdout:344 read 0x2404d56 thru 0x240551a (0x7c5 bytes) 2024-09-16T11:43:37.243 INFO:teuthology.orchestra.run.smithi070.stdout:345 read 0x368a2a2 thru 0x368eaff (0x485e bytes) 2024-09-16T11:43:37.243 INFO:teuthology.orchestra.run.smithi070.stdout:346 punch from 0x752b1d to 0x75392f, (0xe12 bytes) 2024-09-16T11:43:37.243 INFO:teuthology.orchestra.run.smithi070.stdout:347 write 0x769f051 thru 0x76aa400 (0xb3b0 bytes) 2024-09-16T11:43:37.243 INFO:teuthology.orchestra.run.smithi070.stdout:348 read 0x1998cb9 thru 0x19a6e08 (0xe150 bytes) 2024-09-16T11:43:37.243 INFO:teuthology.orchestra.run.smithi070.stdout:349 writesame 0xc298335 thru 0xc2a457b data_size 0x31(0xc247 bytes) 2024-09-16T11:43:37.243 INFO:teuthology.orchestra.run.smithi070.stdout:350 trunc from 0xc2a457c to 0x9ce3f1 2024-09-16T11:43:37.243 INFO:teuthology.orchestra.run.smithi070.stdout:352 trunc from 0x9ce3f1 to 0x206691a 2024-09-16T11:43:37.244 INFO:teuthology.orchestra.run.smithi070.stdout:353 trunc from 0x206691a to 0x4e705ef 2024-09-16T11:43:37.244 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:37.244 INFO:teuthology.orchestra.run.smithi070.stdout:355 write 0x82d4d21 thru 0x82d9801 (0x4ae1 bytes) 2024-09-16T11:43:37.244 INFO:teuthology.orchestra.run.smithi070.stdout:356 punch from 0x1f9dd6 to 0x205589, (0xb7b3 bytes) 2024-09-16T11:43:37.244 INFO:teuthology.orchestra.run.smithi070.stdout:357 read 0x57b069e thru 0x57ba56f (0x9ed2 bytes) 2024-09-16T11:43:37.244 INFO:teuthology.orchestra.run.smithi070.stdout:358 write 0x8e64e9b thru 0x8e6892b (0x3a91 bytes) 2024-09-16T11:43:37.244 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:37.239+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:37.246 INFO:teuthology.orchestra.run.smithi070.stdout:359 read 0x70af912 thru 0x70b2f87 (0x3676 bytes) 2024-09-16T11:43:37.246 INFO:teuthology.orchestra.run.smithi070.stdout:360 punch from 0x767e7e9 to 0x76899ab, (0xb1c2 bytes) 2024-09-16T11:43:37.246 INFO:teuthology.orchestra.run.smithi070.stdout:362 writesame 0x8a52e20 thru 0x8a54079 data_size 0x36(0x125a bytes) 2024-09-16T11:43:37.249 INFO:teuthology.orchestra.run.smithi070.stdout:363 read 0x1c4d74a thru 0x1c5c06e (0xe925 bytes) 2024-09-16T11:43:37.249 INFO:teuthology.orchestra.run.smithi070.stdout:364 read 0x7ecef90 thru 0x7edef20 (0xff91 bytes) 2024-09-16T11:43:37.255 INFO:teuthology.orchestra.run.smithi070.stdout:366 trunc from 0x8e6892c to 0xd38f719 2024-09-16T11:43:37.260 INFO:teuthology.orchestra.run.smithi070.stdout:369 read 0x363009f thru 0x36392d8 (0x923a bytes) 2024-09-16T11:43:37.260 INFO:teuthology.orchestra.run.smithi070.stdout:372 punch from 0x1ae89a9 to 0x1ae9054, (0x6ab bytes) 2024-09-16T11:43:37.260 INFO:teuthology.orchestra.run.smithi070.stdout:373 writesame 0x5157129 thru 0x5164ae1 data_size 0x21(0xd9b9 bytes) 2024-09-16T11:43:37.263 INFO:teuthology.orchestra.run.smithi070.stdout:374 read 0x49beb24 thru 0x49c26d0 (0x3bad bytes) 2024-09-16T11:43:37.263 INFO:teuthology.orchestra.run.smithi070.stdout:377 punch from 0x6e8d228 to 0x6e91e64, (0x4c3c bytes) 2024-09-16T11:43:37.263 INFO:teuthology.orchestra.run.smithi070.stdout:378 write 0x6b63c8b thru 0x6b6d50d (0x9883 bytes) 2024-09-16T11:43:37.267 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:37.267 INFO:teuthology.orchestra.run.smithi070.stdout:382 punch from 0x7b5f3dd to 0x7b61094, (0x1cb7 bytes) 2024-09-16T11:43:37.267 INFO:teuthology.orchestra.run.smithi070.stdout:383 writesame 0x4ed2b24 thru 0x4ed2bc4 data_size 0x1(0xa1 bytes) 2024-09-16T11:43:37.276 INFO:teuthology.orchestra.run.smithi070.stdout:384 trunc from 0xd38f719 to 0x77b0069 2024-09-16T11:43:37.285 INFO:teuthology.orchestra.run.smithi070.stdout:385 write 0x4697d82 thru 0x469abe0 (0x2e5f bytes) 2024-09-16T11:43:37.287 INFO:teuthology.orchestra.run.smithi070.stdout:386 writesame 0x13b019a thru 0x13b698b data_size 0xf(0x67f2 bytes) 2024-09-16T11:43:37.296 INFO:teuthology.orchestra.run.smithi070.stdout:387 trunc from 0x77b0069 to 0x23bd557 2024-09-16T11:43:37.308 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:37.308 INFO:teuthology.orchestra.run.smithi070.stdout:390 punch from 0xd07a0c to 0xd0b886, (0x3e7a bytes) 2024-09-16T11:43:37.322 INFO:teuthology.orchestra.run.smithi070.stdout:393 write 0x8d7c76b thru 0x8d7d9f6 (0x128c bytes) 2024-09-16T11:43:37.324 INFO:teuthology.orchestra.run.smithi070.stdout:395 read 0x80e9585 thru 0x80f5f75 (0xc9f1 bytes) 2024-09-16T11:43:37.325 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:37.325 INFO:teuthology.orchestra.run.smithi070.stdout:397 read 0x8a4a7fa thru 0x8a557ce (0xafd5 bytes) 2024-09-16T11:43:37.325 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:37.325 INFO:teuthology.orchestra.run.smithi070.stdout:399 read 0x5ae3efc thru 0x5aeb465 (0x756a bytes) 2024-09-16T11:43:37.325 INFO:teuthology.orchestra.run.smithi070.stdout:404 write 0x335919f thru 0x335c206 (0x3068 bytes) 2024-09-16T11:43:37.328 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:37.329 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:37.329 INFO:teuthology.orchestra.run.smithi070.stdout:407 writesame 0x7b7a24f thru 0x7b82848 data_size 0xb(0x85fa bytes) 2024-09-16T11:43:37.335 INFO:teuthology.orchestra.run.smithi070.stdout:410 read 0x24402da thru 0x24437c8 (0x34ef bytes) 2024-09-16T11:43:37.335 INFO:teuthology.orchestra.run.smithi070.stdout:413 writesame 0x7bb707e thru 0x7bba753 data_size 0x1(0x36d6 bytes) 2024-09-16T11:43:37.342 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:37.340+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:37.342 INFO:teuthology.orchestra.run.smithi070.stdout:414 read 0x41cdd36 thru 0x41dd86d (0xfb38 bytes) 2024-09-16T11:43:37.342 INFO:teuthology.orchestra.run.smithi070.stdout:415 write 0x2a71817 thru 0x2a815ee (0xfdd8 bytes) 2024-09-16T11:43:37.346 INFO:teuthology.orchestra.run.smithi070.stdout:416 punch from 0x981a7 to 0x9842b, (0x284 bytes) 2024-09-16T11:43:37.346 INFO:teuthology.orchestra.run.smithi070.stdout:417 read 0x2df88e6 thru 0x2e083be (0xfad9 bytes) 2024-09-16T11:43:37.346 INFO:teuthology.orchestra.run.smithi070.stdout:418 writesame 0x18327b3 thru 0x183b9b5 data_size 0x1(0x9203 bytes) 2024-09-16T11:43:37.352 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:37.352 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:37.352 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:37.352 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:37.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:43:37.368 INFO:teuthology.orchestra.run.smithi070.stdout:419 read 0x71ab39 thru 0x723148 (0x8610 bytes) 2024-09-16T11:43:37.368 INFO:teuthology.orchestra.run.smithi070.stdout:420 write 0x6d52974 thru 0x6d53775 (0xe02 bytes) 2024-09-16T11:43:37.371 INFO:teuthology.orchestra.run.smithi070.stdout:421 punch from 0x5736080 to 0x5743d58, (0xdcd8 bytes) 2024-09-16T11:43:37.381 INFO:teuthology.orchestra.run.smithi070.stdout:422 trunc from 0x8d7d9f7 to 0xa9f386 2024-09-16T11:43:37.400 INFO:teuthology.orchestra.run.smithi070.stdout:425 writesame 0x3cd4df thru 0x3d38f1 data_size 0x11(0x6413 bytes) 2024-09-16T11:43:37.401 INFO:teuthology.orchestra.run.smithi070.stdout:427 clone 2 order 20 su 16384 sc 15 2024-09-16T11:43:37.443 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:37.441+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:37.468 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:37.468 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:37.468 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:37.468 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:37.468 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:37.476 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:43:37.560 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:37.558+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:37.614 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:37.614 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:37.614 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:37.614 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:37.614 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.614 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4247.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-16T11:43:37.615 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:14.730801+0000", 2024-09-16T11:43:37.615 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.881615590999999, 2024-09-16T11:43:37.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000654092, 2024-09-16T11:43:37.615 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.615 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.615 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.615 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730801+0000", 2024-09-16T11:43:37.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730801+0000", 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730801+0000", 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9000000000000001e-07 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730807+0000", 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730818+0000", 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0577000000000001e-05 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730827+0000", 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8130000000000002e-06 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730864+0000", 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7249000000000002e-05 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.730887+0000", 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2492000000000001e-05 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.731414+0000", 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00052705899999999997 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:14.731455+0000", 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1155000000000002e-05 2024-09-16T11:43:37.618 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.619 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.619 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.619 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T11:43:37.619 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:33.777581+0000", 2024-09-16T11:43:37.619 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.834835134, 2024-09-16T11:43:37.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001711986, 2024-09-16T11:43:37.619 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.619 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.619 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.619 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.619 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.620 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 10 2024-09-16T11:43:37.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.620 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.620 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.620 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777581+0000", 2024-09-16T11:43:37.620 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.620 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.620 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777581+0000", 2024-09-16T11:43:37.620 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.621 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.621 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777583+0000", 2024-09-16T11:43:37.621 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.584e-06 2024-09-16T11:43:37.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.621 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.621 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777589+0000", 2024-09-16T11:43:37.621 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6309999999999997e-06 2024-09-16T11:43:37.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.621 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.621 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777602+0000", 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2935999999999999e-05 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777612+0000", 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.859e-06 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777658+0000", 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5380000000000003e-05 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777789+0000", 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013106399999999999 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.777992+0000", 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000203764 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.778848+0000", 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00085536599999999998 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:37.623 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779202+0000", 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00035394000000000002 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779253+0000", 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1328000000000001e-05 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779293+0000", 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0133999999999999e-05 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.624 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.625 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.625 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.625 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.4545.0:10 4.1c e31/28 ondisk, result = 0)", 2024-09-16T11:43:37.625 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:33.779134+0000", 2024-09-16T11:43:37.625 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.8332826629999999, 2024-09-16T11:43:37.625 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000169073, 2024-09-16T11:43:37.625 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.625 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:37.625 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.625 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.625 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.626 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779134+0000", 2024-09-16T11:43:37.626 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.626 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.626 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.626 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779134+0000", 2024-09-16T11:43:37.626 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.626 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.626 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.626 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779134+0000", 2024-09-16T11:43:37.626 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.99e-07 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779138+0000", 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8290000000000001e-06 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779147+0000", 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2070000000000001e-06 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779151+0000", 2024-09-16T11:43:37.627 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9729999999999999e-06 2024-09-16T11:43:37.628 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.628 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.628 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.628 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779186+0000", 2024-09-16T11:43:37.628 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5157000000000002e-05 2024-09-16T11:43:37.628 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.628 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.628 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.628 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779195+0000", 2024-09-16T11:43:37.628 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6799999999999999e-06 2024-09-16T11:43:37.628 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.628 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.628 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:33.779303+0000", 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000107528 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:759 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_get_name in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:36.938663+0000", 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.67375305399999996, 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00030183100000000002, 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:37.629 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 759 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.938663+0000", 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.938663+0000", 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.938665+0000", 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7749999999999999e-06 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.938676+0000", 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0519e-05 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.938693+0000", 2024-09-16T11:43:37.631 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7532000000000001e-05 2024-09-16T11:43:37.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.938699+0000", 2024-09-16T11:43:37.632 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8719999999999999e-06 2024-09-16T11:43:37.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.938753+0000", 2024-09-16T11:43:37.632 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4073999999999998e-05 2024-09-16T11:43:37.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.938838+0000", 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5066000000000002e-05 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.938965+0000", 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000126993 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4545.0:928 4.4 e32/28 4:25d4547f:::rbd_data.11c17b8dc4e5.0000000000000009:head v 32'7, mlcod=31'5)", 2024-09-16T11:43:37.633 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.111120+0000", 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.50129637299999996, 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00093304800000000002, 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.111120+0000", 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.111120+0000", 2024-09-16T11:43:37.634 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.111121+0000", 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.114e-06 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.111142+0000", 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1203999999999999e-05 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.635 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.111156+0000", 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4249e-05 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.111163+0000", 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.111215+0000", 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1811999999999998e-05 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.636 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.111241+0000", 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6214999999999999e-05 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.112020+0000", 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00077909300000000002 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.112053+0000", 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3068999999999998e-05 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4545.0:955 4.4 e32/28 4:25d4547f:::rbd_data.11c17b8dc4e5.0000000000000009:head v 32'8, mlcod=31'6)", 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.128905+0000", 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.48351088199999998, 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00106404, 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.128905+0000", 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.638 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.128905+0000", 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.128907+0000", 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.545e-06 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.128950+0000", 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2614000000000002e-05 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.639 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.128967+0000", 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7295999999999998e-05 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.128974+0000", 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1749999999999999e-06 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.129021+0000", 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6746000000000003e-05 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.640 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.641 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.641 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.129044+0000", 2024-09-16T11:43:37.641 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3581999999999999e-05 2024-09-16T11:43:37.641 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.641 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.641 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.641 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.129942+0000", 2024-09-16T11:43:37.641 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00089811100000000005 2024-09-16T11:43:37.641 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.641 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.641 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.641 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.129969+0000", 2024-09-16T11:43:37.641 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6971e-05 2024-09-16T11:43:37.642 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.642 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.642 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.642 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.642 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.642 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4545.0:1001 4.4 e32/28 4:25d4547f:::rbd_data.11c17b8dc4e5.0000000000000009:head v 32'9, mlcod=32'7)", 2024-09-16T11:43:37.642 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.193232+0000", 2024-09-16T11:43:37.642 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.41918470699999999, 2024-09-16T11:43:37.642 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00069024199999999998, 2024-09-16T11:43:37.642 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.642 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.642 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.642 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.643 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.643 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193232+0000", 2024-09-16T11:43:37.643 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.643 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.643 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.643 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193232+0000", 2024-09-16T11:43:37.643 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.643 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.643 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.643 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193233+0000", 2024-09-16T11:43:37.643 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5290000000000001e-06 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193243+0000", 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0103e-05 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193262+0000", 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9165999999999999e-05 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.644 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193270+0000", 2024-09-16T11:43:37.645 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9899999999999997e-06 2024-09-16T11:43:37.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.645 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.645 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.645 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193320+0000", 2024-09-16T11:43:37.645 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0009000000000002e-05 2024-09-16T11:43:37.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.645 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.645 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.645 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193343+0000", 2024-09-16T11:43:37.645 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2798000000000001e-05 2024-09-16T11:43:37.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.645 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193886+0000", 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00054316399999999995 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193922+0000", 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5482999999999999e-05 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.646 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4545.0:1156 4.4 e32/28 4:25d4547f:::rbd_data.11c17b8dc4e5.0000000000000009:head v 32'10, mlcod=32'8)", 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.344171+0000", 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.26824478499999999, 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00087161499999999995, 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.344171+0000", 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.344171+0000", 2024-09-16T11:43:37.647 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.648 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.648 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.648 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.648 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.344173+0000", 2024-09-16T11:43:37.648 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1710000000000001e-06 2024-09-16T11:43:37.648 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.648 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.648 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.648 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.344212+0000", 2024-09-16T11:43:37.648 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9162000000000002e-05 2024-09-16T11:43:37.648 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.648 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.648 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.649 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.344227+0000", 2024-09-16T11:43:37.649 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5248e-05 2024-09-16T11:43:37.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.649 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.649 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.649 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.344233+0000", 2024-09-16T11:43:37.649 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4670000000000003e-06 2024-09-16T11:43:37.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.649 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.649 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.649 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.344278+0000", 2024-09-16T11:43:37.649 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5648000000000001e-05 2024-09-16T11:43:37.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.344295+0000", 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6750999999999999e-05 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.345015+0000", 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00072022199999999996 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.345043+0000", 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7946e-05 2024-09-16T11:43:37.650 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4545.0:1166 4.4 e32/28 4:25d4547f:::rbd_data.11c17b8dc4e5.0000000000000009:head v 32'11, mlcod=32'9)", 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.382225+0000", 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.230190861, 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00064160699999999998, 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382225+0000", 2024-09-16T11:43:37.651 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382225+0000", 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382227+0000", 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2240000000000001e-06 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382243+0000", 2024-09-16T11:43:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6571000000000001e-05 2024-09-16T11:43:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382262+0000", 2024-09-16T11:43:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8419000000000001e-05 2024-09-16T11:43:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382269+0000", 2024-09-16T11:43:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9040000000000003e-06 2024-09-16T11:43:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382316+0000", 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7160999999999998e-05 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382340+0000", 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4502e-05 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382828+0000", 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00048788599999999998 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:37.654 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382867+0000", 2024-09-16T11:43:37.655 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8940000000000003e-05 2024-09-16T11:43:37.655 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.655 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.655 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.655 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:37.655 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:37.655 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:37.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:43:37.657 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:37.655+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:37.757 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:37.756+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:37.795 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:37.795 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:37.796 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:37.796 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:37.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:43:37.844 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone1 from 0xa9f386 (overlap 0x9ce3f1) to 0x6dd2c2 2024-09-16T11:43:37.857 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:37.856+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:37.893 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-16T11:43:37.910 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:37.910 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:37.910 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:37.910 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:37.910 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:37.922 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:43:37.967 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:37.961+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:38.059 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:38.057+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:38.068 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:38.068 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:38.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:38.068 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:38.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.068 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:823 4.11 4:8f5a8a8e:::rbd_data.11c17b8dc4e5.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1572864~31024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.068 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:36.977844+0000", 2024-09-16T11:43:38.068 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.086985412, 2024-09-16T11:43:38.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015973199999999999, 2024-09-16T11:43:38.068 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.068 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.068 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.069 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.069 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.069 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 823 2024-09-16T11:43:38.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.069 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977844+0000", 2024-09-16T11:43:38.069 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977844+0000", 2024-09-16T11:43:38.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977846+0000", 2024-09-16T11:43:38.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6109999999999999e-06 2024-09-16T11:43:38.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977867+0000", 2024-09-16T11:43:38.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1200999999999999e-05 2024-09-16T11:43:38.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977885+0000", 2024-09-16T11:43:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7932e-05 2024-09-16T11:43:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977901+0000", 2024-09-16T11:43:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5688999999999998e-05 2024-09-16T11:43:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.977957+0000", 2024-09-16T11:43:38.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5943000000000003e-05 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.978107+0000", 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015061600000000001 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.978326+0000", 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00021862699999999999 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:38.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.979296+0000", 2024-09-16T11:43:38.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00097002099999999995 2024-09-16T11:43:38.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.979382+0000", 2024-09-16T11:43:38.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5897999999999996e-05 2024-09-16T11:43:38.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.979418+0000", 2024-09-16T11:43:38.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6205999999999999e-05 2024-09-16T11:43:38.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.979442+0000", 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3575999999999999e-05 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:840 4.11 4:8f5a8a8e:::rbd_data.11c17b8dc4e5.000000000000001e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:36.999598+0000", 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.0652315569999999, 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0030579449999999998, 2024-09-16T11:43:38.074 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 840 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999598+0000", 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999598+0000", 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999599+0000", 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4000000000000001e-07 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999605+0000", 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4649999999999999e-06 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999617+0000", 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1654000000000001e-05 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999623+0000", 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7649999999999999e-06 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999662+0000", 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8875000000000003e-05 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999735+0000", 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3403000000000002e-05 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:36.999805+0000", 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9906999999999993e-05 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.000591+0000", 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00078583199999999998 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.002605+0000", 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020143589999999999 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.002635+0000", 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9547999999999999e-05 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.002656+0000", 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1396999999999999e-05 2024-09-16T11:43:38.079 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.080 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.080 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.080 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:865 4.17 4:ebaef169:::rbd_data.11c17b8dc4e5.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2831334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.080 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.012244+0000", 2024-09-16T11:43:38.080 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.052585224, 2024-09-16T11:43:38.080 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0024779810000000002, 2024-09-16T11:43:38.080 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.080 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.080 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.080 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.081 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.081 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 865 2024-09-16T11:43:38.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.081 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.012244+0000", 2024-09-16T11:43:38.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.012244+0000", 2024-09-16T11:43:38.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.012246+0000", 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.841e-06 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.012253+0000", 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.268e-06 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.012268+0000", 2024-09-16T11:43:38.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5903e-05 2024-09-16T11:43:38.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.083 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.083 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.083 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.012274+0000", 2024-09-16T11:43:38.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6470000000000003e-06 2024-09-16T11:43:38.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.083 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.083 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.083 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.012322+0000", 2024-09-16T11:43:38.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7769999999999998e-05 2024-09-16T11:43:38.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.084 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.012422+0000", 2024-09-16T11:43:38.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000100046 2024-09-16T11:43:38.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.084 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:38.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.012528+0000", 2024-09-16T11:43:38.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000106253 2024-09-16T11:43:38.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:38.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.012941+0000", 2024-09-16T11:43:38.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00041234699999999997 2024-09-16T11:43:38.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.014667+0000", 2024-09-16T11:43:38.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017267389999999999 2024-09-16T11:43:38.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.014700+0000", 2024-09-16T11:43:38.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3198000000000003e-05 2024-09-16T11:43:38.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.014722+0000", 2024-09-16T11:43:38.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1968999999999999e-05 2024-09-16T11:43:38.086 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.086 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.086 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.086 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4545.0:905 4.1a e32/28 4:5b7d482a:::rbd_data.11c17b8dc4e5.0000000000000008:head v 32'9, mlcod=31'7)", 2024-09-16T11:43:38.086 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.077756+0000", 2024-09-16T11:43:38.086 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.98707337100000003, 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017371610000000001, 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.077756+0000", 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.077756+0000", 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.077757+0000", 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2200000000000002e-07 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.078627+0000", 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000869559 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.078639+0000", 2024-09-16T11:43:38.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2044e-05 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.078643+0000", 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5549999999999996e-06 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.078702+0000", 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9079000000000001e-05 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.078726+0000", 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3663999999999999e-05 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.079458+0000", 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00073170799999999997 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.079493+0000", 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5630000000000003e-05 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.091 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1002 4.11 4:8a86a6ca:::rbd_data.11c17b8dc4e5.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.091 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.192970+0000", 2024-09-16T11:43:38.091 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.87185933500000001, 2024-09-16T11:43:38.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0014695260000000001, 2024-09-16T11:43:38.091 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.091 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.091 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.091 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.091 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.091 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1002 2024-09-16T11:43:38.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.091 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.192970+0000", 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.192970+0000", 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.192971+0000", 2024-09-16T11:43:38.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.13e-07 2024-09-16T11:43:38.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.093 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.093 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.192973+0000", 2024-09-16T11:43:38.093 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5050000000000002e-06 2024-09-16T11:43:38.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.093 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.093 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.192980+0000", 2024-09-16T11:43:38.093 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9949999999999999e-06 2024-09-16T11:43:38.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.093 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.192984+0000", 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9709999999999998e-06 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193044+0000", 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9369e-05 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193131+0000", 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7485999999999996e-05 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193244+0000", 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112841 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.194043+0000", 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00079913300000000005 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.194388+0000", 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00034484000000000001 2024-09-16T11:43:38.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.194416+0000", 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7931000000000001e-05 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.194440+0000", 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3941999999999999e-05 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1005 4.11 4:8f5a8a8e:::rbd_data.11c17b8dc4e5.000000000000001e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.193002+0000", 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.87182748200000004, 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021709400000000001, 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1005 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.097 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193002+0000", 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193002+0000", 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193003+0000", 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1199999999999998e-07 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.098 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193005+0000", 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4739999999999999e-06 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193019+0000", 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3407000000000001e-05 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193023+0000", 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3089999999999997e-06 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193510+0000", 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00048679300000000001 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193581+0000", 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1829000000000006e-05 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.193659+0000", 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7459999999999994e-05 2024-09-16T11:43:38.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.101 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:38.101 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.194348+0000", 2024-09-16T11:43:38.101 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00068917800000000003 2024-09-16T11:43:38.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.101 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.101 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.195114+0000", 2024-09-16T11:43:38.101 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00076616399999999995 2024-09-16T11:43:38.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.101 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.101 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.195139+0000", 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4955000000000001e-05 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.195173+0000", 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3958999999999998e-05 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1017 4.0 4:01a40311:::rbd_data.11c1e24d7fee.0000000000000000:4 [sparse-read 2359296~57654] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.196719+0000", 2024-09-16T11:43:38.102 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.86811020699999997, 2024-09-16T11:43:38.103 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015723919999999999, 2024-09-16T11:43:38.103 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.103 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:38.103 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.103 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.103 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.103 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1017 2024-09-16T11:43:38.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.103 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.103 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196719+0000", 2024-09-16T11:43:38.103 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.104 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.104 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196719+0000", 2024-09-16T11:43:38.104 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.104 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.104 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196720+0000", 2024-09-16T11:43:38.104 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8999999999999998e-07 2024-09-16T11:43:38.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196721+0000", 2024-09-16T11:43:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.485e-06 2024-09-16T11:43:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196727+0000", 2024-09-16T11:43:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2040000000000002e-06 2024-09-16T11:43:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196731+0000", 2024-09-16T11:43:38.106 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:43:38.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.106 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.106 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.197820+0000", 2024-09-16T11:43:38.106 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001089051 2024-09-16T11:43:38.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.106 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.106 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.197886+0000", 2024-09-16T11:43:38.106 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6013000000000006e-05 2024-09-16T11:43:38.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.107 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.107 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.198292+0000", 2024-09-16T11:43:38.107 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00040552399999999999 2024-09-16T11:43:38.107 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.107 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.107 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.107 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.107 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1018 4.0 4:01a40311:::rbd_data.11c1e24d7fee.0000000000000000:4 [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.107 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.196735+0000", 2024-09-16T11:43:38.107 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.86809442199999998, 2024-09-16T11:43:38.107 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001746926, 2024-09-16T11:43:38.107 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.108 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:38.108 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.108 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.108 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.108 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1018 2024-09-16T11:43:38.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.108 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.108 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.108 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196735+0000", 2024-09-16T11:43:38.108 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.109 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.109 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196735+0000", 2024-09-16T11:43:38.109 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.109 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.109 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196736+0000", 2024-09-16T11:43:38.109 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.41e-07 2024-09-16T11:43:38.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.109 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.109 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196737+0000", 2024-09-16T11:43:38.109 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:43:38.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.110 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.110 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196743+0000", 2024-09-16T11:43:38.110 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1140000000000002e-06 2024-09-16T11:43:38.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.110 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.110 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196747+0000", 2024-09-16T11:43:38.110 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5930000000000002e-06 2024-09-16T11:43:38.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.110 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.111 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.198330+0000", 2024-09-16T11:43:38.111 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001583428 2024-09-16T11:43:38.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.111 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.111 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.198375+0000", 2024-09-16T11:43:38.111 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4156000000000002e-05 2024-09-16T11:43:38.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.111 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.111 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.198482+0000", 2024-09-16T11:43:38.111 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000107694 2024-09-16T11:43:38.112 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.112 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.112 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.112 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1024 4.0 4:01a40311:::rbd_data.11c1e24d7fee.0000000000000000:4 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.112 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.196780+0000", 2024-09-16T11:43:38.112 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.868049336, 2024-09-16T11:43:38.112 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0019270929999999999, 2024-09-16T11:43:38.112 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.112 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:38.112 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1024 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196780+0000", 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196780+0000", 2024-09-16T11:43:38.113 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196781+0000", 2024-09-16T11:43:38.114 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:43:38.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196784+0000", 2024-09-16T11:43:38.114 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.903e-06 2024-09-16T11:43:38.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.115 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.115 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196791+0000", 2024-09-16T11:43:38.115 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.08e-06 2024-09-16T11:43:38.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.115 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.115 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196794+0000", 2024-09-16T11:43:38.115 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.546e-06 2024-09-16T11:43:38.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.115 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.115 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.198520+0000", 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001726032 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.198597+0000", 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6891999999999995e-05 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.198707+0000", 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011023600000000001 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1025 4.0 4:01a40311:::rbd_data.11c1e24d7fee.0000000000000000:4 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.196799+0000", 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.86803106500000005, 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020559490000000001, 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1025 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.117 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196799+0000", 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196799+0000", 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196799+0000", 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2800000000000003e-07 2024-09-16T11:43:38.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196801+0000", 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.703e-06 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196806+0000", 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196853+0000", 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6278e-05 2024-09-16T11:43:38.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.120 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.120 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.198732+0000", 2024-09-16T11:43:38.120 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001878836 2024-09-16T11:43:38.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.120 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.120 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.198782+0000", 2024-09-16T11:43:38.120 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0649999999999998e-05 2024-09-16T11:43:38.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.120 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.120 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.198855+0000", 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2356999999999999e-05 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1026 4.0 4:01a40311:::rbd_data.11c1e24d7fee.0000000000000000:4 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.196859+0000", 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.86797023699999998, 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021262719999999998, 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.121 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1026 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196859+0000", 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196859+0000", 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.122 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196860+0000", 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5899999999999996e-07 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196863+0000", 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1609999999999999e-06 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196872+0000", 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0990000000000005e-06 2024-09-16T11:43:38.123 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.124 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.124 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.124 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196882+0000", 2024-09-16T11:43:38.124 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7690000000000008e-06 2024-09-16T11:43:38.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.124 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.124 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.124 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.198885+0000", 2024-09-16T11:43:38.124 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002003445 2024-09-16T11:43:38.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.124 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.124 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.124 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.198927+0000", 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1319999999999997e-05 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.198986+0000", 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8919000000000003e-05 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1027 4.0 4:01a40311:::rbd_data.11c1e24d7fee.0000000000000000:4 [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.196886+0000", 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.86794349000000004, 2024-09-16T11:43:38.125 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022425290000000001, 2024-09-16T11:43:38.126 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.126 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:38.126 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.126 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.126 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.126 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1027 2024-09-16T11:43:38.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.126 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.126 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.126 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.126 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196886+0000", 2024-09-16T11:43:38.126 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196886+0000", 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196887+0000", 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196889+0000", 2024-09-16T11:43:38.127 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9489999999999999e-06 2024-09-16T11:43:38.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.128 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.128 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196894+0000", 2024-09-16T11:43:38.128 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:43:38.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.128 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.128 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.196898+0000", 2024-09-16T11:43:38.128 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7510000000000002e-06 2024-09-16T11:43:38.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.129 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.129 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.199010+0000", 2024-09-16T11:43:38.129 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021115890000000001 2024-09-16T11:43:38.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.129 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.129 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.199060+0000", 2024-09-16T11:43:38.129 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0501000000000003e-05 2024-09-16T11:43:38.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.129 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.129 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.199129+0000", 2024-09-16T11:43:38.129 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8717e-05 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1098 4.11 4:88295b20:::rbd_data.11c17b8dc4e5.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.260595+0000", 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.804234318, 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015138720000000001, 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.130 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.131 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1098 2024-09-16T11:43:38.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.131 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.131 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.131 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.260595+0000", 2024-09-16T11:43:38.131 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.131 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.131 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.260595+0000", 2024-09-16T11:43:38.131 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.260597+0000", 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1150000000000001e-06 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.260610+0000", 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2104e-05 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.132 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.260630+0000", 2024-09-16T11:43:38.133 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0322999999999998e-05 2024-09-16T11:43:38.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.133 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.133 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.260638+0000", 2024-09-16T11:43:38.133 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9379999999999992e-06 2024-09-16T11:43:38.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.133 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.133 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.260683+0000", 2024-09-16T11:43:38.133 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5562999999999997e-05 2024-09-16T11:43:38.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.134 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.134 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.260746+0000", 2024-09-16T11:43:38.134 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2395000000000001e-05 2024-09-16T11:43:38.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.134 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:38.134 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.260951+0000", 2024-09-16T11:43:38.134 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00020533300000000001 2024-09-16T11:43:38.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.134 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.134 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.261967+0000", 2024-09-16T11:43:38.134 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001015915 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.262047+0000", 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9585999999999993e-05 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.262077+0000", 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9915000000000001e-05 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.262109+0000", 2024-09-16T11:43:38.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2685000000000003e-05 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1153 4.11 4:8f5a8a8e:::rbd_data.11c17b8dc4e5.000000000000001e:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.334953+0000", 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.729876682, 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0066010399999999999, 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.136 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.137 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1153 2024-09-16T11:43:38.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.137 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.334953+0000", 2024-09-16T11:43:38.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.334953+0000", 2024-09-16T11:43:38.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.141 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.141 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.334954+0000", 2024-09-16T11:43:38.141 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5300000000000003e-07 2024-09-16T11:43:38.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.141 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.141 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.334959+0000", 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0359999999999997e-06 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.334972+0000", 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3089e-05 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.334981+0000", 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5669999999999995e-06 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.335010+0000", 2024-09-16T11:43:38.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9176e-05 2024-09-16T11:43:38.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.335131+0000", 2024-09-16T11:43:38.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000120883 2024-09-16T11:43:38.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:38.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.336045+0000", 2024-09-16T11:43:38.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00091456600000000001 2024-09-16T11:43:38.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.341454+0000", 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.005408904 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.341501+0000", 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7219999999999999e-05 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.341531+0000", 2024-09-16T11:43:38.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9502e-05 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.341554+0000", 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3244e-05 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1165 4.17 4:ebaef169:::rbd_data.11c17b8dc4e5.0000000000000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.381857+0000", 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.68297238299999996, 2024-09-16T11:43:38.145 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0067670730000000002, 2024-09-16T11:43:38.146 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.146 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.146 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.146 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.146 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.146 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1165 2024-09-16T11:43:38.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.146 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.146 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.146 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381857+0000", 2024-09-16T11:43:38.146 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381857+0000", 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381859+0000", 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0810000000000001e-06 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381866+0000", 2024-09-16T11:43:38.147 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0740000000000004e-06 2024-09-16T11:43:38.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.148 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.148 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381885+0000", 2024-09-16T11:43:38.148 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8742000000000001e-05 2024-09-16T11:43:38.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.148 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.148 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381897+0000", 2024-09-16T11:43:38.148 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1576999999999999e-05 2024-09-16T11:43:38.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.148 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381947+0000", 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9722999999999998e-05 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382028+0000", 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1099000000000004e-05 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382151+0000", 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012370299999999999 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382910+0000", 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00075841700000000005 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.388529+0000", 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0056195819999999997 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.388587+0000", 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7979999999999997e-05 2024-09-16T11:43:38.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.388624+0000", 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7095000000000003e-05 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1167 4.11 4:8a86a6ca:::rbd_data.11c17b8dc4e5.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.382397+0000", 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.68243260100000003, 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.006237884, 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.151 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1167 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382397+0000", 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382397+0000", 2024-09-16T11:43:38.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382398+0000", 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1699999999999998e-07 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382402+0000", 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8700000000000002e-06 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.153 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382411+0000", 2024-09-16T11:43:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2720000000000003e-06 2024-09-16T11:43:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382418+0000", 2024-09-16T11:43:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8390000000000001e-06 2024-09-16T11:43:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382482+0000", 2024-09-16T11:43:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4418000000000006e-05 2024-09-16T11:43:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382558+0000", 2024-09-16T11:43:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5939999999999995e-05 2024-09-16T11:43:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382657+0000", 2024-09-16T11:43:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8455999999999994e-05 2024-09-16T11:43:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.383419+0000", 2024-09-16T11:43:38.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00076257300000000001 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.388556+0000", 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0051369579999999996 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.388601+0000", 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4598999999999998e-05 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.388635+0000", 2024-09-16T11:43:38.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4341999999999997e-05 2024-09-16T11:43:38.157 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.157 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.157 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.157 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1171 4.11 4:8f5a8a8e:::rbd_data.11c17b8dc4e5.000000000000001e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.157 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.382443+0000", 2024-09-16T11:43:38.157 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.68238635199999997, 2024-09-16T11:43:38.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0062635570000000003, 2024-09-16T11:43:38.157 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.157 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.157 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.158 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.158 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.158 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1171 2024-09-16T11:43:38.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.158 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.158 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.158 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382443+0000", 2024-09-16T11:43:38.158 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.158 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382443+0000", 2024-09-16T11:43:38.159 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.159 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382444+0000", 2024-09-16T11:43:38.159 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6699999999999999e-07 2024-09-16T11:43:38.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.159 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382447+0000", 2024-09-16T11:43:38.159 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7930000000000002e-06 2024-09-16T11:43:38.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382454+0000", 2024-09-16T11:43:38.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:43:38.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382458+0000", 2024-09-16T11:43:38.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0230000000000003e-06 2024-09-16T11:43:38.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382945+0000", 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00048784000000000002 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.383036+0000", 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0879999999999997e-05 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.383140+0000", 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103238 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.161 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:38.162 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.383672+0000", 2024-09-16T11:43:38.162 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000532618 2024-09-16T11:43:38.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.162 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.162 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.388663+0000", 2024-09-16T11:43:38.162 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004991166 2024-09-16T11:43:38.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.162 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.162 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.388688+0000", 2024-09-16T11:43:38.162 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.472e-05 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.388707+0000", 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8862e-05 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1187 4.1d 4:b8712c40:::rbd_data.11c17b8dc4e5.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.399460+0000", 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.66536965000000003, 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0014915429999999999, 2024-09-16T11:43:38.163 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.164 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.164 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.164 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.164 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.164 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1187 2024-09-16T11:43:38.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.164 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.164 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.164 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.164 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.399460+0000", 2024-09-16T11:43:38.164 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.164 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.399460+0000", 2024-09-16T11:43:38.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.399461+0000", 2024-09-16T11:43:38.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.207e-06 2024-09-16T11:43:38.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.399467+0000", 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7769999999999999e-06 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.399480+0000", 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2563e-05 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.399484+0000", 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.399517+0000", 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2716999999999999e-05 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.399574+0000", 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7466000000000002e-05 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.399753+0000", 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017829900000000001 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:38.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.400780+0000", 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001027189 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.400886+0000", 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000106288 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.400913+0000", 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6970000000000001e-05 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.168 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.400952+0000", 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8495999999999999e-05 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1279 4.11 4:8cd0e658:::rbd_data.11c17b8dc4e5.0000000000000002:head [stat,truncate 1835008] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.851001+0000", 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.21382905699999999, 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002343085, 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.169 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1279 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851001+0000", 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851001+0000", 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851002+0000", 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3710000000000001e-06 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851007+0000", 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1789999999999999e-06 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851019+0000", 2024-09-16T11:43:38.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1438e-05 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851025+0000", 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9109999999999998e-06 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851063+0000", 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8359000000000003e-05 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851129+0000", 2024-09-16T11:43:38.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6427999999999995e-05 2024-09-16T11:43:38.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:38.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851242+0000", 2024-09-16T11:43:38.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112376 2024-09-16T11:43:38.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:38.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851923+0000", 2024-09-16T11:43:38.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00068174200000000005 2024-09-16T11:43:38.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.853296+0000", 2024-09-16T11:43:38.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0013723380000000001 2024-09-16T11:43:38.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.853323+0000", 2024-09-16T11:43:38.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7294999999999999e-05 2024-09-16T11:43:38.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.175 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.175 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.853344+0000", 2024-09-16T11:43:38.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0647999999999999e-05 2024-09-16T11:43:38.175 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.175 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.175 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.175 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1280 4.1d 4:b8712c40:::rbd_data.11c17b8dc4e5.0000000000000003:head [stat,truncate 1692354] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T11:43:38.175 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.851048+0000", 2024-09-16T11:43:38.175 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.21378133599999999, 2024-09-16T11:43:38.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015368999999999999, 2024-09-16T11:43:38.176 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:38.176 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.176 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:38.176 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:38.176 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.176 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1280 2024-09-16T11:43:38.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.176 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:38.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:38.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851048+0000", 2024-09-16T11:43:38.176 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:38.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851048+0000", 2024-09-16T11:43:38.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:38.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:38.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851049+0000", 2024-09-16T11:43:38.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:43:38.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851052+0000", 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.405e-06 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851060+0000", 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5129999999999999e-06 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851064+0000", 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3560000000000003e-06 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851115+0000", 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0553000000000002e-05 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851164+0000", 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9178000000000002e-05 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.851274+0000", 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000110043 2024-09-16T11:43:38.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.852167+0000", 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00089337900000000003 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.852497+0000", 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000330007 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.852544+0000", 2024-09-16T11:43:38.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6171999999999998e-05 2024-09-16T11:43:38.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:38.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:38.181 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:38.181 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.852585+0000", 2024-09-16T11:43:38.181 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1644000000000003e-05 2024-09-16T11:43:38.181 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.181 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.181 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.181 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:38.181 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:38.181 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:38.183 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:43:38.184 INFO:teuthology.orchestra.run.smithi070.stdout:428 writesame 0xda738c4 thru 0xda7a0a6 data_size 0x2d(0x67e3 bytes) 2024-09-16T11:43:38.185 INFO:teuthology.orchestra.run.smithi070.stdout:429 read 0xa4b72c0 thru 0xa4bc292 (0x4fd3 bytes) 2024-09-16T11:43:38.185 INFO:teuthology.orchestra.run.smithi070.stdout:430 trunc from 0xda7a0a7 to 0xb249743 2024-09-16T11:43:38.185 INFO:teuthology.orchestra.run.smithi070.stdout:431 punch from 0xa6c7f7d to 0xa6c9201, (0x1284 bytes) 2024-09-16T11:43:38.185 INFO:teuthology.orchestra.run.smithi070.stdout:432 read 0x216ae84 thru 0x216f905 (0x4a82 bytes) 2024-09-16T11:43:38.185 INFO:teuthology.orchestra.run.smithi070.stdout:435 read 0x236d7c2 thru 0x2375478 (0x7cb7 bytes) 2024-09-16T11:43:38.185 INFO:teuthology.orchestra.run.smithi070.stdout:436 writesame 0xbbf3601 thru 0xbc018af data_size 0x1(0xe2af bytes) 2024-09-16T11:43:38.185 INFO:teuthology.orchestra.run.smithi070.stdout:437 write 0xb3c8927 thru 0xb3cebbb (0x6295 bytes) 2024-09-16T11:43:38.185 INFO:teuthology.orchestra.run.smithi070.stdout:438 read 0x87c082c thru 0x87c0bf2 (0x3c7 bytes) 2024-09-16T11:43:38.185 INFO:teuthology.orchestra.run.smithi070.stdout:truncating to largest ever: 0xeacd08c 2024-09-16T11:43:38.185 INFO:teuthology.orchestra.run.smithi070.stdout:439 trunc from 0xbc018b0 to 0xeacd08c 2024-09-16T11:43:38.185 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:38.185 INFO:teuthology.orchestra.run.smithi070.stdout:441 trunc from 0xeacd08c to 0x581f64f 2024-09-16T11:43:38.186 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:38.180+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:38.187 INFO:teuthology.orchestra.run.smithi070.stdout:442 writesame 0xbbb674e thru 0xbbc1af6 data_size 0x1(0xb3a9 bytes) 2024-09-16T11:43:38.187 INFO:teuthology.orchestra.run.smithi070.stdout:443 writesame 0x32f25d0 thru 0x32ff388 data_size 0x5(0xcdb9 bytes) 2024-09-16T11:43:38.196 INFO:teuthology.orchestra.run.smithi070.stdout:444 trunc from 0xbbc1af7 to 0xa30c53e 2024-09-16T11:43:38.202 INFO:teuthology.orchestra.run.smithi070.stdout:446 read 0x334a108 thru 0x334ddff (0x3cf8 bytes) 2024-09-16T11:43:38.207 INFO:teuthology.orchestra.run.smithi070.stdout:447 trunc from 0xa30c53e to 0x57d2ed9 2024-09-16T11:43:38.214 INFO:teuthology.orchestra.run.smithi070.stdout:448 trunc from 0x57d2ed9 to 0x3e97cd8 2024-09-16T11:43:38.226 INFO:teuthology.orchestra.run.smithi070.stdout:449 writesame 0x4609414 thru 0x460d4b8 data_size 0x1(0x40a5 bytes) 2024-09-16T11:43:38.240 INFO:teuthology.orchestra.run.smithi070.stdout:450 write 0xafb9467 thru 0xafbbee4 (0x2a7e bytes) 2024-09-16T11:43:38.243 INFO:teuthology.orchestra.run.smithi070.stdout:451 read 0xa270bb thru 0xa27cc6 (0xc0c bytes) 2024-09-16T11:43:38.244 INFO:teuthology.orchestra.run.smithi070.stdout:452 writesame 0x6b78445 thru 0x6b7afc8 data_size 0xa(0x2b84 bytes) 2024-09-16T11:43:38.246 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:38.246 INFO:teuthology.orchestra.run.smithi070.stdout:456 write 0x12297f4 thru 0x12365ea (0xcdf7 bytes) 2024-09-16T11:43:38.253 INFO:teuthology.orchestra.run.smithi070.stdout:460 read 0x2e2df59 thru 0x2e37ce0 (0x9d88 bytes) 2024-09-16T11:43:38.253 INFO:teuthology.orchestra.run.smithi070.stdout:461 write 0x3e4a172 thru 0x3e53219 (0x90a8 bytes) 2024-09-16T11:43:38.257 INFO:teuthology.orchestra.run.smithi070.stdout:462 read 0xf1617c thru 0xf1a90a (0x478f bytes) 2024-09-16T11:43:38.265 INFO:teuthology.orchestra.run.smithi070.stdout:463 trunc from 0xafbbee5 to 0x4c95b89 2024-09-16T11:43:38.274 INFO:teuthology.orchestra.run.smithi070.stdout:464 punch from 0x37551e4 to 0x3761bbe, (0xc9da bytes) 2024-09-16T11:43:38.274 INFO:teuthology.orchestra.run.smithi070.stdout:465 read 0x39226b2 thru 0x392377f (0x10ce bytes) 2024-09-16T11:43:38.283 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:38.282+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:38.289 INFO:teuthology.orchestra.run.smithi070.stdout:466 write 0xda1e6db thru 0xda285a8 (0x9ece bytes) 2024-09-16T11:43:38.289 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:38.289 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:38.289 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:43:38.290 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:38.294 INFO:teuthology.orchestra.run.smithi070.stdout:467 write 0xb2b333b thru 0xb2c2a17 (0xf6dd bytes) 2024-09-16T11:43:38.299 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:43:38.302 INFO:teuthology.orchestra.run.smithi070.stdout:468 writesame 0x94e8df6 thru 0x94f0697 data_size 0x2(0x78a2 bytes) 2024-09-16T11:43:38.309 INFO:teuthology.orchestra.run.smithi070.stdout:469 read 0xa84124b thru 0xa84960b (0x83c1 bytes) 2024-09-16T11:43:38.309 INFO:teuthology.orchestra.run.smithi070.stdout:470 read 0xb3a965e thru 0xb3b27d2 (0x9175 bytes) 2024-09-16T11:43:38.309 INFO:teuthology.orchestra.run.smithi070.stdout:471 read 0x38933fe thru 0x389965a (0x625d bytes) 2024-09-16T11:43:38.310 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:38.310 INFO:teuthology.orchestra.run.smithi070.stdout:473 read 0x280689e thru 0x280ef09 (0x866c bytes) 2024-09-16T11:43:38.310 INFO:teuthology.orchestra.run.smithi070.stdout:475 write 0xa1ecbdf thru 0xa1fb909 (0xed2b bytes) 2024-09-16T11:43:38.320 INFO:teuthology.orchestra.run.smithi070.stdout:476 trunc from 0xda285a9 to 0x88f6147 2024-09-16T11:43:38.347 INFO:teuthology.orchestra.run.smithi070.stdout:477 writesame 0xe0c57e5 thru 0xe0cd067 data_size 0x1(0x7883 bytes) 2024-09-16T11:43:38.347 INFO:teuthology.orchestra.run.smithi070.stdout:478 write 0x2a711ba thru 0x2a7ad6e (0x9bb5 bytes) 2024-09-16T11:43:38.351 INFO:teuthology.orchestra.run.smithi070.stdout:479 write 0x239b581 thru 0x23a741d (0xbe9d bytes) 2024-09-16T11:43:38.355 INFO:teuthology.orchestra.run.smithi070.stdout:481 read 0xc6fabcc thru 0xc706449 (0xb87e bytes) 2024-09-16T11:43:38.356 INFO:teuthology.orchestra.run.smithi070.stdout:484 punch from 0x8cedf17 to 0x8ceee44, (0xf2d bytes) 2024-09-16T11:43:38.356 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:38.356 INFO:teuthology.orchestra.run.smithi070.stdout:486 punch from 0x3d9ca36 to 0x3da858d, (0xbb57 bytes) 2024-09-16T11:43:38.356 INFO:teuthology.orchestra.run.smithi070.stdout:487 punch from 0x8c32da1 to 0x8c3d295, (0xa4f4 bytes) 2024-09-16T11:43:38.356 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:38.356 INFO:teuthology.orchestra.run.smithi070.stdout:490 write 0x617ac6f thru 0x617aec3 (0x255 bytes) 2024-09-16T11:43:38.358 INFO:teuthology.orchestra.run.smithi070.stdout:491 read 0x16c05e2 thru 0x16c79ed (0x740c bytes) 2024-09-16T11:43:38.358 INFO:teuthology.orchestra.run.smithi070.stdout:492 write 0xdf756b9 thru 0xdf82935 (0xd27d bytes) 2024-09-16T11:43:38.363 INFO:teuthology.orchestra.run.smithi070.stdout:493 clone 3 order 23 su 1048576 sc 14 2024-09-16T11:43:38.384 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:38.382+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:38.439 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:38.439 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:38.439 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:38.439 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:38.439 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:38.447 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:43:38.485 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:38.483+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:38.547 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:38.547 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:43:38.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:43:38.547 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:43:38.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.547 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:860 4.e 4:71e4238f:::rbd_object_map.11c17b8dc4e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.547 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.005961+0000", 2024-09-16T11:43:38.547 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.537792375, 2024-09-16T11:43:38.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035273800000000001, 2024-09-16T11:43:38.547 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.548 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.548 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.548 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.548 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.548 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 860 2024-09-16T11:43:38.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.548 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.005961+0000", 2024-09-16T11:43:38.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.005961+0000", 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.005962+0000", 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4200000000000005e-07 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.005967+0000", 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1329999999999998e-06 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.005978+0000", 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0746e-05 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.005983+0000", 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5289999999999999e-06 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.006031+0000", 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8217000000000003e-05 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.006084+0000", 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2961999999999999e-05 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.006274+0000", 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000189574 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.009073+0000", 2024-09-16T11:43:38.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0027994119999999998 2024-09-16T11:43:38.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.009430+0000", 2024-09-16T11:43:38.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00035663399999999999 2024-09-16T11:43:38.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.009462+0000", 2024-09-16T11:43:38.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1760000000000001e-05 2024-09-16T11:43:38.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.009488+0000", 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6571e-05 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:861 4.e 4:71e4238f:::rbd_object_map.11c17b8dc4e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.006808+0000", 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.5369447859999998, 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052776810000000002, 2024-09-16T11:43:38.553 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.554 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.554 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 861 2024-09-16T11:43:38.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.554 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.006808+0000", 2024-09-16T11:43:38.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.006808+0000", 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.006809+0000", 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799999999999996e-07 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.006813+0000", 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9790000000000004e-06 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.006822+0000", 2024-09-16T11:43:38.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8959999999999992e-06 2024-09-16T11:43:38.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.006828+0000", 2024-09-16T11:43:38.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:43:38.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.006986+0000", 2024-09-16T11:43:38.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015744700000000001 2024-09-16T11:43:38.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.007036+0000", 2024-09-16T11:43:38.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0691e-05 2024-09-16T11:43:38.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.007244+0000", 2024-09-16T11:43:38.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020716300000000001 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.009510+0000", 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022663510000000002 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.012009+0000", 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024992 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.012052+0000", 2024-09-16T11:43:38.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2552999999999997e-05 2024-09-16T11:43:38.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.012086+0000", 2024-09-16T11:43:38.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4471000000000003e-05 2024-09-16T11:43:38.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.559 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.559 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:863 4.e 4:71e4238f:::rbd_object_map.11c17b8dc4e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.560 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.007474+0000", 2024-09-16T11:43:38.560 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.536279159, 2024-09-16T11:43:38.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046951930000000003, 2024-09-16T11:43:38.560 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.560 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.560 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 863 2024-09-16T11:43:38.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.560 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.007474+0000", 2024-09-16T11:43:38.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.007474+0000", 2024-09-16T11:43:38.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.007475+0000", 2024-09-16T11:43:38.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5499999999999997e-07 2024-09-16T11:43:38.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.007480+0000", 2024-09-16T11:43:38.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6029999999999998e-06 2024-09-16T11:43:38.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.007490+0000", 2024-09-16T11:43:38.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0291e-05 2024-09-16T11:43:38.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.007495+0000", 2024-09-16T11:43:38.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0189999999999997e-06 2024-09-16T11:43:38.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.007555+0000", 2024-09-16T11:43:38.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0285000000000003e-05 2024-09-16T11:43:38.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.007630+0000", 2024-09-16T11:43:38.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4812000000000001e-05 2024-09-16T11:43:38.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.007851+0000", 2024-09-16T11:43:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022098300000000001 2024-09-16T11:43:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.009518+0000", 2024-09-16T11:43:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016671170000000001 2024-09-16T11:43:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.012118+0000", 2024-09-16T11:43:38.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002599629 2024-09-16T11:43:38.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.012144+0000", 2024-09-16T11:43:38.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6812999999999999e-05 2024-09-16T11:43:38.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.012169+0000", 2024-09-16T11:43:38.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4786e-05 2024-09-16T11:43:38.566 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.566 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.566 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.566 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:868 4.e 4:71e4238f:::rbd_object_map.11c17b8dc4e5:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.566 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.017402+0000", 2024-09-16T11:43:38.566 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.5263507569999999, 2024-09-16T11:43:38.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002628128, 2024-09-16T11:43:38.566 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.566 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.567 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 868 2024-09-16T11:43:38.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.567 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.017402+0000", 2024-09-16T11:43:38.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.017402+0000", 2024-09-16T11:43:38.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.017403+0000", 2024-09-16T11:43:38.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4399999999999999e-07 2024-09-16T11:43:38.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.017407+0000", 2024-09-16T11:43:38.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8820000000000003e-06 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.017415+0000", 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1710000000000002e-06 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.017419+0000", 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9110000000000003e-06 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.017443+0000", 2024-09-16T11:43:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3839999999999999e-05 2024-09-16T11:43:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.017490+0000", 2024-09-16T11:43:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6675999999999998e-05 2024-09-16T11:43:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.017628+0000", 2024-09-16T11:43:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000138749 2024-09-16T11:43:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.019926+0000", 2024-09-16T11:43:38.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022979390000000001 2024-09-16T11:43:38.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.019984+0000", 2024-09-16T11:43:38.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7902000000000002e-05 2024-09-16T11:43:38.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.020011+0000", 2024-09-16T11:43:38.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7154000000000001e-05 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.020031+0000", 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.916e-05 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:893 4.e 4:71e4238f:::rbd_object_map.11c17b8dc4e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.051404+0000", 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.4923489540000001, 2024-09-16T11:43:38.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037469669999999999, 2024-09-16T11:43:38.573 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.573 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.573 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 893 2024-09-16T11:43:38.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.573 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.051404+0000", 2024-09-16T11:43:38.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.051404+0000", 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.051405+0000", 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.051406+0000", 2024-09-16T11:43:38.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.747e-06 2024-09-16T11:43:38.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.051410+0000", 2024-09-16T11:43:38.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:43:38.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.051412+0000", 2024-09-16T11:43:38.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1019999999999999e-06 2024-09-16T11:43:38.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.051934+0000", 2024-09-16T11:43:38.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00052234000000000004 2024-09-16T11:43:38.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.052017+0000", 2024-09-16T11:43:38.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3001e-05 2024-09-16T11:43:38.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.052397+0000", 2024-09-16T11:43:38.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00037956300000000002 2024-09-16T11:43:38.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.054996+0000", 2024-09-16T11:43:38.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025989310000000001 2024-09-16T11:43:38.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.055089+0000", 2024-09-16T11:43:38.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3071000000000004e-05 2024-09-16T11:43:38.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.055117+0000", 2024-09-16T11:43:38.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8685000000000001e-05 2024-09-16T11:43:38.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.055151+0000", 2024-09-16T11:43:38.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3824e-05 2024-09-16T11:43:38.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.578 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.579 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:897 4.e 4:71e4238f:::rbd_object_map.11c17b8dc4e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.579 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.058426+0000", 2024-09-16T11:43:38.579 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.4853268370000001, 2024-09-16T11:43:38.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050868939999999998, 2024-09-16T11:43:38.579 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.579 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.579 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 897 2024-09-16T11:43:38.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.579 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.058426+0000", 2024-09-16T11:43:38.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.058426+0000", 2024-09-16T11:43:38.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.058427+0000", 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4799999999999999e-07 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.058429+0000", 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3020000000000002e-06 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.058436+0000", 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8249999999999999e-06 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.058439+0000", 2024-09-16T11:43:38.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5329999999999999e-06 2024-09-16T11:43:38.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.058964+0000", 2024-09-16T11:43:38.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00052437999999999996 2024-09-16T11:43:38.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.059042+0000", 2024-09-16T11:43:38.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8436999999999995e-05 2024-09-16T11:43:38.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.059407+0000", 2024-09-16T11:43:38.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00036482499999999999 2024-09-16T11:43:38.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.063314+0000", 2024-09-16T11:43:38.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039067379999999999 2024-09-16T11:43:38.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.063437+0000", 2024-09-16T11:43:38.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012360800000000001 2024-09-16T11:43:38.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.063483+0000", 2024-09-16T11:43:38.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5062999999999999e-05 2024-09-16T11:43:38.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.063513+0000", 2024-09-16T11:43:38.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0735000000000003e-05 2024-09-16T11:43:38.584 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.584 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.585 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:905 4.1a 4:5b7d482a:::rbd_data.11c17b8dc4e5.0000000000000008:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.585 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.071040+0000", 2024-09-16T11:43:38.585 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.472713106, 2024-09-16T11:43:38.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.017484133999999998, 2024-09-16T11:43:38.585 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.585 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.585 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 905 2024-09-16T11:43:38.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.586 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071040+0000", 2024-09-16T11:43:38.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071040+0000", 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071041+0000", 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9299999999999998e-07 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071044+0000", 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071053+0000", 2024-09-16T11:43:38.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9940000000000006e-06 2024-09-16T11:43:38.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071057+0000", 2024-09-16T11:43:38.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0990000000000001e-06 2024-09-16T11:43:38.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071087+0000", 2024-09-16T11:43:38.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9311000000000001e-05 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071157+0000", 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0207000000000001e-05 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.074097+0000", 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0029405410000000001 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.088459+0000", 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014361237000000001 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.088465+0000", 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8369999999999999e-06 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.088497+0000", 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2048000000000002e-05 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.088524+0000", 2024-09-16T11:43:38.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6767e-05 2024-09-16T11:43:38.591 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.591 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.591 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.591 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4545.0:905 4.1a e32/28 ondisk, result = 0)", 2024-09-16T11:43:38.591 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.082713+0000", 2024-09-16T11:43:38.591 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.461039746, 2024-09-16T11:43:38.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057468270000000004, 2024-09-16T11:43:38.591 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.591 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.082713+0000", 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.082713+0000", 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.082714+0000", 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9800000000000003e-07 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.082717+0000", 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.151e-06 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.082728+0000", 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0773e-05 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.082734+0000", 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0739999999999998e-06 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.088448+0000", 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057138730000000004 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.088455+0000", 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6470000000000001e-06 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.088460+0000", 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:915 4.e 4:71e4238f:::rbd_object_map.11c17b8dc4e5:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.097118+0000", 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.446634886, 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003466925, 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.595 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.596 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 915 2024-09-16T11:43:38.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.596 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.097118+0000", 2024-09-16T11:43:38.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.097118+0000", 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.097119+0000", 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.097126+0000", 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4400000000000002e-06 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.097136+0000", 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0494e-05 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.097140+0000", 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0840000000000002e-06 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.097172+0000", 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1801999999999998e-05 2024-09-16T11:43:38.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.097220+0000", 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7777e-05 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.097362+0000", 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014222900000000001 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.097863+0000", 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00050155200000000005 2024-09-16T11:43:38.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.100538+0000", 2024-09-16T11:43:38.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026747490000000001 2024-09-16T11:43:38.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.100566+0000", 2024-09-16T11:43:38.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7988999999999999e-05 2024-09-16T11:43:38.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.100585+0000", 2024-09-16T11:43:38.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9049999999999999e-05 2024-09-16T11:43:38.601 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.601 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.601 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.601 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:1057 4.13 4:cd968791:::rbd_header.11c17b8dc4e5:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.601 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.217362+0000", 2024-09-16T11:43:38.601 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.3263913110000001, 2024-09-16T11:43:38.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026168329999999998, 2024-09-16T11:43:38.601 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.601 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1057 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.217362+0000", 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.217362+0000", 2024-09-16T11:43:38.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.217363+0000", 2024-09-16T11:43:38.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0100000000000004e-07 2024-09-16T11:43:38.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.217366+0000", 2024-09-16T11:43:38.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-16T11:43:38.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.217382+0000", 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6277e-05 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.217389+0000", 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9110000000000004e-06 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.217421+0000", 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1745999999999997e-05 2024-09-16T11:43:38.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.217468+0000", 2024-09-16T11:43:38.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7113000000000001e-05 2024-09-16T11:43:38.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.217570+0000", 2024-09-16T11:43:38.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010174 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.218021+0000", 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000451137 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.219919+0000", 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018979579999999999 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.219946+0000", 2024-09-16T11:43:38.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7161e-05 2024-09-16T11:43:38.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.219979+0000", 2024-09-16T11:43:38.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2549000000000003e-05 2024-09-16T11:43:38.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.607 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.607 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:1150 4.e 4:71e4238f:::rbd_object_map.11c17b8dc4e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.608 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.328223+0000", 2024-09-16T11:43:38.608 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.2155300389999999, 2024-09-16T11:43:38.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050807969999999997, 2024-09-16T11:43:38.608 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.608 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.608 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1150 2024-09-16T11:43:38.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.609 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.328223+0000", 2024-09-16T11:43:38.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.328223+0000", 2024-09-16T11:43:38.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.328224+0000", 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1799999999999995e-07 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.328226+0000", 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6139999999999999e-06 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.328231+0000", 2024-09-16T11:43:38.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5079999999999999e-06 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.328234+0000", 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.328783+0000", 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054863899999999996 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.328827+0000", 2024-09-16T11:43:38.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4620000000000003e-05 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.329024+0000", 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197176 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.333176+0000", 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041520309999999996 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.333257+0000", 2024-09-16T11:43:38.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0898999999999999e-05 2024-09-16T11:43:38.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.333284+0000", 2024-09-16T11:43:38.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6889e-05 2024-09-16T11:43:38.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.333304+0000", 2024-09-16T11:43:38.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9853000000000001e-05 2024-09-16T11:43:38.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.613 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:1158 4.e 4:71e4238f:::rbd_object_map.11c17b8dc4e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.345567+0000", 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.1981858400000001, 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0042290419999999997, 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1158 2024-09-16T11:43:38.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.345567+0000", 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.345567+0000", 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.345569+0000", 2024-09-16T11:43:38.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.345579+0000", 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6569999999999992e-06 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.345604+0000", 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5176000000000001e-05 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.345615+0000", 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0753e-05 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.345706+0000", 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1636000000000002e-05 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.345930+0000", 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000223942 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.346333+0000", 2024-09-16T11:43:38.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00040269200000000001 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.347008+0000", 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00067530499999999996 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.349714+0000", 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0027058300000000002 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.349758+0000", 2024-09-16T11:43:38.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3480999999999999e-05 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.349796+0000", 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8720000000000002e-05 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:1168 4.5 e32/28 4:a11716bb:::rbd_data.11c17b8dc4e5.000000000000000d:head v 32'8, mlcod=32'6)", 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.382080+0000", 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.1616735330000001, 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031358789999999998, 2024-09-16T11:43:38.619 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.620 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.620 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382080+0000", 2024-09-16T11:43:38.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382080+0000", 2024-09-16T11:43:38.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382080+0000", 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.13e-07 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382085+0000", 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0220000000000004e-06 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382095+0000", 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4739999999999994e-06 2024-09-16T11:43:38.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382098+0000", 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382134+0000", 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5979000000000002e-05 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382172+0000", 2024-09-16T11:43:38.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7647000000000001e-05 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.385170+0000", 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0029980979999999998 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.385216+0000", 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5683000000000003e-05 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:1191 4.13 4:cd968791:::rbd_header.11c17b8dc4e5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.787127+0000", 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.75662631999999996, 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002875252, 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1191 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.787127+0000", 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.787127+0000", 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.787128+0000", 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4190000000000001e-06 2024-09-16T11:43:38.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.787134+0000", 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8710000000000002e-06 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.787148+0000", 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3461999999999999e-05 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.787154+0000", 2024-09-16T11:43:38.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0639999999999999e-06 2024-09-16T11:43:38.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.787196+0000", 2024-09-16T11:43:38.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2765000000000001e-05 2024-09-16T11:43:38.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-16T11:43:38.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.787202+0000", 2024-09-16T11:43:38.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8869999999999997e-06 2024-09-16T11:43:38.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.788875+0000", 2024-09-16T11:43:38.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001672874 2024-09-16T11:43:38.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.788932+0000", 2024-09-16T11:43:38.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6424000000000001e-05 2024-09-16T11:43:38.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.789142+0000", 2024-09-16T11:43:38.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021063700000000001 2024-09-16T11:43:38.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.789706+0000", 2024-09-16T11:43:38.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000563198 2024-09-16T11:43:38.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.789950+0000", 2024-09-16T11:43:38.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024419099999999999 2024-09-16T11:43:38.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.789979+0000", 2024-09-16T11:43:38.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9128999999999998e-05 2024-09-16T11:43:38.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.790002+0000", 2024-09-16T11:43:38.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3331000000000001e-05 2024-09-16T11:43:38.630 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.630 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.630 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.631 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:1274 4.e 4:71e4238f:::rbd_object_map.11c17b8dc4e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T11:43:38.631 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.844067+0000", 2024-09-16T11:43:38.631 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.69968637300000003, 2024-09-16T11:43:38.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039771939999999999, 2024-09-16T11:43:38.631 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.631 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.631 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1274 2024-09-16T11:43:38.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.632 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844067+0000", 2024-09-16T11:43:38.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844067+0000", 2024-09-16T11:43:38.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844068+0000", 2024-09-16T11:43:38.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:43:38.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844072+0000", 2024-09-16T11:43:38.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9740000000000004e-06 2024-09-16T11:43:38.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844082+0000", 2024-09-16T11:43:38.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.005e-05 2024-09-16T11:43:38.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844088+0000", 2024-09-16T11:43:38.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8560000000000003e-06 2024-09-16T11:43:38.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844835+0000", 2024-09-16T11:43:38.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074759399999999997 2024-09-16T11:43:38.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844920+0000", 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5321000000000005e-05 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.845190+0000", 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026911900000000001 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.847907+0000", 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0027173729999999999 2024-09-16T11:43:38.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.847982+0000", 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4598999999999995e-05 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.848012+0000", 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0198000000000001e-05 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.848044+0000", 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2310000000000001e-05 2024-09-16T11:43:38.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.637 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.637 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.637 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:1275 4.e 4:71e4238f:::rbd_object_map.11c17b8dc4e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T11:43:38.637 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.844094+0000", 2024-09-16T11:43:38.637 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.699659366, 2024-09-16T11:43:38.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067668939999999999, 2024-09-16T11:43:38.637 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.637 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1275 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844094+0000", 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844094+0000", 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844095+0000", 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844100+0000", 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1229999999999999e-06 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844108+0000", 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3899999999999993e-06 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844113+0000", 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9269999999999996e-06 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.845471+0000", 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001358197 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.845554+0000", 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3015000000000004e-05 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.845781+0000", 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022718099999999999 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.850742+0000", 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049611400000000002 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.850791+0000", 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8767999999999998e-05 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.850833+0000", 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1714000000000001e-05 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.850861+0000", 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7789000000000001e-05 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.642 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:1276 4.e 4:71e4238f:::rbd_object_map.11c17b8dc4e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T11:43:38.643 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.844186+0000", 2024-09-16T11:43:38.643 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.69956684099999999, 2024-09-16T11:43:38.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067440629999999998, 2024-09-16T11:43:38.643 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.643 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.643 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1276 2024-09-16T11:43:38.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.643 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844186+0000", 2024-09-16T11:43:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844186+0000", 2024-09-16T11:43:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844187+0000", 2024-09-16T11:43:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6300000000000005e-07 2024-09-16T11:43:38.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844191+0000", 2024-09-16T11:43:38.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7230000000000002e-06 2024-09-16T11:43:38.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844202+0000", 2024-09-16T11:43:38.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1494e-05 2024-09-16T11:43:38.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.844208+0000", 2024-09-16T11:43:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5729999999999998e-06 2024-09-16T11:43:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.846166+0000", 2024-09-16T11:43:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019586019999999998 2024-09-16T11:43:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.846216+0000", 2024-09-16T11:43:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0059000000000003e-05 2024-09-16T11:43:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.846411+0000", 2024-09-16T11:43:38.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00019485999999999999 2024-09-16T11:43:38.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.850749+0000", 2024-09-16T11:43:38.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0043376029999999998 2024-09-16T11:43:38.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.850884+0000", 2024-09-16T11:43:38.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013536700000000001 2024-09-16T11:43:38.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.850910+0000", 2024-09-16T11:43:38.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5213999999999999e-05 2024-09-16T11:43:38.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.850930+0000", 2024-09-16T11:43:38.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905000000000001e-05 2024-09-16T11:43:38.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.648 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.649 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:1277 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:head [stat,truncate 1835008] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T11:43:38.649 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.846670+0000", 2024-09-16T11:43:38.649 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.69708320199999996, 2024-09-16T11:43:38.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041634569999999997, 2024-09-16T11:43:38.649 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.649 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1277 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.846670+0000", 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.846670+0000", 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.846671+0000", 2024-09-16T11:43:38.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7599999999999996e-07 2024-09-16T11:43:38.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.846675+0000", 2024-09-16T11:43:38.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5029999999999999e-06 2024-09-16T11:43:38.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.846689+0000", 2024-09-16T11:43:38.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3118e-05 2024-09-16T11:43:38.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.846697+0000", 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0120000000000005e-06 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.846725+0000", 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8317e-05 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.846769+0000", 2024-09-16T11:43:38.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4277999999999999e-05 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.846864+0000", 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4450999999999994e-05 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.848597+0000", 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001733479 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.850772+0000", 2024-09-16T11:43:38.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002174549 2024-09-16T11:43:38.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.850801+0000", 2024-09-16T11:43:38.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9641999999999998e-05 2024-09-16T11:43:38.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.850833+0000", 2024-09-16T11:43:38.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2231999999999999e-05 2024-09-16T11:43:38.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.654 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:1575 4.1a 4:5d1e9f11:::rbd_data.11c130c94979.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.303904+0000", 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.239849019, 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003144619, 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1575 2024-09-16T11:43:38.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.303904+0000", 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.303904+0000", 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.303905+0000", 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-07 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.303909+0000", 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.98e-06 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.303918+0000", 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6679999999999996e-06 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.303935+0000", 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6413e-05 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.303985+0000", 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0019999999999999e-05 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.304111+0000", 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012648999999999999 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-16T11:43:38.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.304698+0000", 2024-09-16T11:43:38.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00058644800000000005 2024-09-16T11:43:38.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.306962+0000", 2024-09-16T11:43:38.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022639729999999999 2024-09-16T11:43:38.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.306986+0000", 2024-09-16T11:43:38.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4545e-05 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.307015+0000", 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8790999999999999e-05 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.307049+0000", 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3664999999999997e-05 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.661 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:1599 4.e 4:71eeaec0:::rbd_data.11c130c94979.00000000000000ad:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T11:43:38.661 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.321461+0000", 2024-09-16T11:43:38.661 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.22229252599999999, 2024-09-16T11:43:38.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003501732, 2024-09-16T11:43:38.661 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:38.661 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:38.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:38.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:38.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:38.661 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1599 2024-09-16T11:43:38.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.661 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:38.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:38.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.321461+0000", 2024-09-16T11:43:38.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:38.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.321461+0000", 2024-09-16T11:43:38.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:38.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:38.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.321461+0000", 2024-09-16T11:43:38.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.46e-07 2024-09-16T11:43:38.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:38.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.321465+0000", 2024-09-16T11:43:38.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9280000000000003e-06 2024-09-16T11:43:38.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:38.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.321469+0000", 2024-09-16T11:43:38.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.284e-06 2024-09-16T11:43:38.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:38.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.321472+0000", 2024-09-16T11:43:38.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.571e-06 2024-09-16T11:43:38.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:38.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.321522+0000", 2024-09-16T11:43:38.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0467999999999999e-05 2024-09-16T11:43:38.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:38.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.321594+0000", 2024-09-16T11:43:38.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1388000000000002e-05 2024-09-16T11:43:38.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:38.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.321695+0000", 2024-09-16T11:43:38.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101055 2024-09-16T11:43:38.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.322408+0000", 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071350700000000005 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.324915+0000", 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025062119999999998 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.324942+0000", 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7282999999999999e-05 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:38.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:38.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.324962+0000", 2024-09-16T11:43:38.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-05 2024-09-16T11:43:38.667 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.667 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.667 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.667 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:38.667 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:38.667 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:38.669 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:43:38.669 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:38.667+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:38.769 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:38.767+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:38.799 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:43:38.811 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:38.811 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:38.811 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:43:38.811 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:38.820 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:43:38.856 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone2 from 0xe0cd068 (overlap 0xa9f386) to 0x0 2024-09-16T11:43:38.904 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:38.903+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:38.971 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:38.969+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:38.974 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-16T11:43:38.977 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:38.978 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:38.978 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:38.978 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:38.978 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:38.987 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:43:38.988 INFO:teuthology.orchestra.run.smithi070.stdout:494 read 0xdf946ea thru 0xdfa33ba (0xecd1 bytes) 2024-09-16T11:43:38.988 INFO:teuthology.orchestra.run.smithi070.stdout:495 writesame 0x10b607c thru 0x10c3738 data_size 0x1(0xd6bd bytes) 2024-09-16T11:43:39.021 INFO:teuthology.orchestra.run.smithi070.stdout:497 read 0xc05d6e1 thru 0xc06adbe (0xd6de bytes) 2024-09-16T11:43:39.022 INFO:teuthology.orchestra.run.smithi070.stdout:500 read 0x2783281 thru 0x2785c99 (0x2a19 bytes) 2024-09-16T11:43:39.022 INFO:teuthology.orchestra.run.smithi070.stdout:503 write 0x5d7439d thru 0x5d7b91f (0x7583 bytes) 2024-09-16T11:43:39.027 INFO:teuthology.orchestra.run.smithi070.stdout:506 writesame 0x7c9b6c8 thru 0x7ca3acd data_size 0x6(0x8406 bytes) 2024-09-16T11:43:39.032 INFO:teuthology.orchestra.run.smithi070.stdout:507 read 0x620a9b8 thru 0x620bfd9 (0x1622 bytes) 2024-09-16T11:43:39.032 INFO:teuthology.orchestra.run.smithi070.stdout:509 read 0x71b8f4 thru 0x725ef5 (0xa602 bytes) 2024-09-16T11:43:39.032 INFO:teuthology.orchestra.run.smithi070.stdout:511 write 0x2dc1f2b thru 0x2dce5d4 (0xc6aa bytes) 2024-09-16T11:43:39.041 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:43:39.041 INFO:teuthology.orchestra.run.smithi070.stdout:512 clone 4 order 20 su 32768 sc 4 2024-09-16T11:43:39.084 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:39.082+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:39.128 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:39.128 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:43:39.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:43:39.128 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:43:39.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.128 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2119 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2375680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.128 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864670+0000", 2024-09-16T11:43:39.128 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.26086605099999999, 2024-09-16T11:43:39.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013064175000000001, 2024-09-16T11:43:39.129 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.129 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.129 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2119 2024-09-16T11:43:39.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.129 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864670+0000", 2024-09-16T11:43:39.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864670+0000", 2024-09-16T11:43:39.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864670+0000", 2024-09-16T11:43:39.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:43:39.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864672+0000", 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.691e-06 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864677+0000", 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6860000000000003e-06 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864681+0000", 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-06 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877539+0000", 2024-09-16T11:43:39.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012858748999999999 2024-09-16T11:43:39.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877632+0000", 2024-09-16T11:43:39.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2367999999999994e-05 2024-09-16T11:43:39.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877734+0000", 2024-09-16T11:43:39.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102 2024-09-16T11:43:39.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.133 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.133 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2120 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.133 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864685+0000", 2024-09-16T11:43:39.133 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.26085106299999999, 2024-09-16T11:43:39.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013171611999999999, 2024-09-16T11:43:39.133 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.134 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.134 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2120 2024-09-16T11:43:39.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.134 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864685+0000", 2024-09-16T11:43:39.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864685+0000", 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864685+0000", 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864692+0000", 2024-09-16T11:43:39.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5039999999999999e-06 2024-09-16T11:43:39.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864697+0000", 2024-09-16T11:43:39.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.699e-06 2024-09-16T11:43:39.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864701+0000", 2024-09-16T11:43:39.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6009999999999999e-06 2024-09-16T11:43:39.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877756+0000", 2024-09-16T11:43:39.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0130554 2024-09-16T11:43:39.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877801+0000", 2024-09-16T11:43:39.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5157000000000002e-05 2024-09-16T11:43:39.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877856+0000", 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4910000000000001e-05 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2121 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2818048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864708+0000", 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.26082802100000002, 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013354938, 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2121 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864708+0000", 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864708+0000", 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864708+0000", 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864710+0000", 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864716+0000", 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7289999999999997e-06 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864720+0000", 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9990000000000002e-06 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877883+0000", 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013162903 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877969+0000", 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6027999999999997e-05 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878063+0000", 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4086999999999996e-05 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2122 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864723+0000", 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.26081277200000003, 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013470271000000001, 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.142 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.143 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2122 2024-09-16T11:43:39.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.143 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864723+0000", 2024-09-16T11:43:39.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864723+0000", 2024-09-16T11:43:39.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864723+0000", 2024-09-16T11:43:39.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6199999999999999e-07 2024-09-16T11:43:39.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864725+0000", 2024-09-16T11:43:39.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5960000000000001e-06 2024-09-16T11:43:39.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864730+0000", 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1950000000000002e-06 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864733+0000", 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-06 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878093+0000", 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013359473 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878140+0000", 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6692e-05 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878193+0000", 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3683000000000001e-05 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2124 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2326528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864738+0000", 2024-09-16T11:43:39.146 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.26079797799999999, 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013679180000000001, 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2124 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864738+0000", 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864738+0000", 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864738+0000", 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864740+0000", 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5990000000000001e-06 2024-09-16T11:43:39.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864745+0000", 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5600000000000001e-06 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864749+0000", 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2590000000000001e-06 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878230+0000", 2024-09-16T11:43:39.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013481215 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878314+0000", 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4037999999999997e-05 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878417+0000", 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010315900000000001 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.151 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2125 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2523136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.151 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864752+0000", 2024-09-16T11:43:39.151 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.26078387400000003, 2024-09-16T11:43:39.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013813690999999999, 2024-09-16T11:43:39.151 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.151 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.151 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2125 2024-09-16T11:43:39.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.151 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864752+0000", 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864752+0000", 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864752+0000", 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864754+0000", 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864760+0000", 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5119999999999997e-06 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864764+0000", 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5779999999999999e-06 2024-09-16T11:43:39.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878448+0000", 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013683749 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878496+0000", 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8028999999999997e-05 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878566+0000", 2024-09-16T11:43:39.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9900000000000005e-05 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2126 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2719744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864767+0000", 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.26076869899999999, 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013991063999999999, 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.156 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2126 2024-09-16T11:43:39.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.156 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864767+0000", 2024-09-16T11:43:39.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864767+0000", 2024-09-16T11:43:39.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864767+0000", 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1800000000000002e-07 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864769+0000", 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.53e-06 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864772+0000", 2024-09-16T11:43:39.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4580000000000002e-06 2024-09-16T11:43:39.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864774+0000", 2024-09-16T11:43:39.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.979e-06 2024-09-16T11:43:39.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878598+0000", 2024-09-16T11:43:39.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013823821999999999 2024-09-16T11:43:39.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878673+0000", 2024-09-16T11:43:39.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4476999999999999e-05 2024-09-16T11:43:39.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878758+0000", 2024-09-16T11:43:39.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-05 2024-09-16T11:43:39.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.162 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.162 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2182 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 65536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.162 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.865997+0000", 2024-09-16T11:43:39.162 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.25953891499999998, 2024-09-16T11:43:39.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012930596000000001, 2024-09-16T11:43:39.162 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.162 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.163 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2182 2024-09-16T11:43:39.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.163 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865997+0000", 2024-09-16T11:43:39.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865997+0000", 2024-09-16T11:43:39.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865998+0000", 2024-09-16T11:43:39.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:43:39.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866001+0000", 2024-09-16T11:43:39.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7220000000000001e-06 2024-09-16T11:43:39.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866013+0000", 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.188e-05 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866026+0000", 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2938999999999999e-05 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878795+0000", 2024-09-16T11:43:39.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012769024 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878864+0000", 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8427000000000001e-05 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878927+0000", 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3746999999999998e-05 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.167 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2183 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 262144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.167 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866033+0000", 2024-09-16T11:43:39.167 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.25950229800000002, 2024-09-16T11:43:39.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013099107, 2024-09-16T11:43:39.167 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.167 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.167 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2183 2024-09-16T11:43:39.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.167 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866033+0000", 2024-09-16T11:43:39.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866033+0000", 2024-09-16T11:43:39.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866034+0000", 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866036+0000", 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866047+0000", 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1083999999999999e-05 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866062+0000", 2024-09-16T11:43:39.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5156e-05 2024-09-16T11:43:39.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878961+0000", 2024-09-16T11:43:39.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012898687000000001 2024-09-16T11:43:39.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879036+0000", 2024-09-16T11:43:39.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4540000000000001e-05 2024-09-16T11:43:39.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879133+0000", 2024-09-16T11:43:39.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6887999999999998e-05 2024-09-16T11:43:39.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.171 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.171 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2184 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.172 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866067+0000", 2024-09-16T11:43:39.172 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.25946828900000002, 2024-09-16T11:43:39.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013274744, 2024-09-16T11:43:39.172 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.172 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.172 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2184 2024-09-16T11:43:39.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.172 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866067+0000", 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866067+0000", 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866068+0000", 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:43:39.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866070+0000", 2024-09-16T11:43:39.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-16T11:43:39.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866076+0000", 2024-09-16T11:43:39.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1809999999999998e-06 2024-09-16T11:43:39.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866080+0000", 2024-09-16T11:43:39.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8279999999999996e-06 2024-09-16T11:43:39.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879170+0000", 2024-09-16T11:43:39.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013089747000000001 2024-09-16T11:43:39.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879252+0000", 2024-09-16T11:43:39.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2279000000000003e-05 2024-09-16T11:43:39.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879342+0000", 2024-09-16T11:43:39.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0195e-05 2024-09-16T11:43:39.176 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.176 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.176 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.176 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2185 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.176 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866084+0000", 2024-09-16T11:43:39.176 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.25945192299999997, 2024-09-16T11:43:39.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013513447, 2024-09-16T11:43:39.177 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.177 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.177 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2185 2024-09-16T11:43:39.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.177 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866084+0000", 2024-09-16T11:43:39.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866084+0000", 2024-09-16T11:43:39.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866084+0000", 2024-09-16T11:43:39.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-09-16T11:43:39.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866086+0000", 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6789999999999999e-06 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866091+0000", 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.626e-06 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866095+0000", 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:43:39.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879406+0000", 2024-09-16T11:43:39.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013311523 2024-09-16T11:43:39.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879497+0000", 2024-09-16T11:43:39.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0507999999999995e-05 2024-09-16T11:43:39.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879597+0000", 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010027900000000001 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2186 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866100+0000", 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.259435888, 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013635298000000001, 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2186 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866100+0000", 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866100+0000", 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866100+0000", 2024-09-16T11:43:39.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.15e-07 2024-09-16T11:43:39.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866109+0000", 2024-09-16T11:43:39.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8189999999999998e-06 2024-09-16T11:43:39.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866115+0000", 2024-09-16T11:43:39.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2770000000000002e-06 2024-09-16T11:43:39.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866121+0000", 2024-09-16T11:43:39.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1279999999999996e-06 2024-09-16T11:43:39.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879618+0000", 2024-09-16T11:43:39.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013496869 2024-09-16T11:43:39.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879676+0000", 2024-09-16T11:43:39.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8075000000000002e-05 2024-09-16T11:43:39.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879735+0000", 2024-09-16T11:43:39.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8715000000000003e-05 2024-09-16T11:43:39.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.185 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.186 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2187 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.186 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866125+0000", 2024-09-16T11:43:39.186 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.25941022400000002, 2024-09-16T11:43:39.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013740278999999999, 2024-09-16T11:43:39.186 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.186 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.186 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2187 2024-09-16T11:43:39.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866125+0000", 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866125+0000", 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866126+0000", 2024-09-16T11:43:39.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.08e-07 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866128+0000", 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6160000000000001e-06 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866145+0000", 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.747e-05 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866161+0000", 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6382e-05 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879767+0000", 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013605894 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879813+0000", 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999999e-05 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879866+0000", 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3109000000000003e-05 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2188 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1540096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866166+0000", 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.25936988900000002, 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013814610999999999, 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2188 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866166+0000", 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866166+0000", 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866166+0000", 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866181+0000", 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.489e-05 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866185+0000", 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.386e-06 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866188+0000", 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.469e-06 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879888+0000", 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013700273000000001 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879932+0000", 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4039000000000003e-05 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879980+0000", 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8186000000000002e-05 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2189 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866191+0000", 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.25934520900000002, 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013900292999999999, 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2189 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866191+0000", 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866191+0000", 2024-09-16T11:43:39.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866191+0000", 2024-09-16T11:43:39.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1699999999999999e-07 2024-09-16T11:43:39.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866193+0000", 2024-09-16T11:43:39.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3410000000000001e-06 2024-09-16T11:43:39.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866196+0000", 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9840000000000001e-06 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866198+0000", 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8369999999999999e-06 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880001+0000", 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013803426000000001 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880042+0000", 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0410000000000001e-05 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880091+0000", 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8977999999999998e-05 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.198 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2190 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1933312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.199 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866200+0000", 2024-09-16T11:43:39.199 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.25933593100000002, 2024-09-16T11:43:39.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013997562, 2024-09-16T11:43:39.199 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.199 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.199 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2190 2024-09-16T11:43:39.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.199 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866200+0000", 2024-09-16T11:43:39.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866200+0000", 2024-09-16T11:43:39.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866200+0000", 2024-09-16T11:43:39.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1899999999999999e-07 2024-09-16T11:43:39.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866201+0000", 2024-09-16T11:43:39.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0249999999999999e-06 2024-09-16T11:43:39.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866204+0000", 2024-09-16T11:43:39.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.858e-06 2024-09-16T11:43:39.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866206+0000", 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880111+0000", 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013905463999999999 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880151+0000", 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0182999999999998e-05 2024-09-16T11:43:39.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880197+0000", 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6050000000000001e-05 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2191 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2129920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866207+0000", 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.259328277, 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014099525, 2024-09-16T11:43:39.203 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.204 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.204 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.204 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.204 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.204 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2191 2024-09-16T11:43:39.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.204 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866207+0000", 2024-09-16T11:43:39.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866207+0000", 2024-09-16T11:43:39.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866208+0000", 2024-09-16T11:43:39.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-16T11:43:39.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866209+0000", 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.519e-06 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866212+0000", 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8210000000000002e-06 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866218+0000", 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5280000000000003e-06 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880218+0000", 2024-09-16T11:43:39.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014000383999999999 2024-09-16T11:43:39.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880259+0000", 2024-09-16T11:43:39.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0815000000000001e-05 2024-09-16T11:43:39.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880307+0000", 2024-09-16T11:43:39.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8241000000000001e-05 2024-09-16T11:43:39.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.208 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.208 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2192 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2572288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.208 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866249+0000", 2024-09-16T11:43:39.208 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.25928717600000001, 2024-09-16T11:43:39.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014184179999999999, 2024-09-16T11:43:39.209 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.209 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.209 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2192 2024-09-16T11:43:39.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.209 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866249+0000", 2024-09-16T11:43:39.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866249+0000", 2024-09-16T11:43:39.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866249+0000", 2024-09-16T11:43:39.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:43:39.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866251+0000", 2024-09-16T11:43:39.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1119999999999999e-06 2024-09-16T11:43:39.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866257+0000", 2024-09-16T11:43:39.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9159999999999998e-06 2024-09-16T11:43:39.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866262+0000", 2024-09-16T11:43:39.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8910000000000003e-06 2024-09-16T11:43:39.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880330+0000", 2024-09-16T11:43:39.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014068429 2024-09-16T11:43:39.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880383+0000", 2024-09-16T11:43:39.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029000000000003e-05 2024-09-16T11:43:39.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880433+0000", 2024-09-16T11:43:39.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9446000000000001e-05 2024-09-16T11:43:39.213 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.213 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.213 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.213 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2193 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2768896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.213 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866266+0000", 2024-09-16T11:43:39.213 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.25926932899999999, 2024-09-16T11:43:39.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.01427954, 2024-09-16T11:43:39.213 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.213 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:39.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.214 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.214 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2193 2024-09-16T11:43:39.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.214 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866266+0000", 2024-09-16T11:43:39.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866266+0000", 2024-09-16T11:43:39.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866267+0000", 2024-09-16T11:43:39.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7300000000000001e-07 2024-09-16T11:43:39.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866269+0000", 2024-09-16T11:43:39.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.73e-06 2024-09-16T11:43:39.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866275+0000", 2024-09-16T11:43:39.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0229999999999998e-06 2024-09-16T11:43:39.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866278+0000", 2024-09-16T11:43:39.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-16T11:43:39.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880454+0000", 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014175755999999999 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880498+0000", 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4033999999999999e-05 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880546+0000", 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7849999999999998e-05 2024-09-16T11:43:39.217 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.218 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.218 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2670 4.15 4:aecbca69:::rbd_data.11c197cf718c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:39.218 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.995522+0000", 2024-09-16T11:43:39.218 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.130013252, 2024-09-16T11:43:39.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024883367999999999, 2024-09-16T11:43:39.218 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:39.218 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:39.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:39.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:39.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2670 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.995522+0000", 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.995522+0000", 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:39.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.995524+0000", 2024-09-16T11:43:39.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2810000000000001e-06 2024-09-16T11:43:39.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:39.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996603+0000", 2024-09-16T11:43:39.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010795329999999999 2024-09-16T11:43:39.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:39.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996616+0000", 2024-09-16T11:43:39.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2850999999999999e-05 2024-09-16T11:43:39.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996626+0000", 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0362e-05 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996664+0000", 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7078e-05 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996757+0000", 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3614000000000002e-05 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.001077+0000", 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0043196570000000002 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.020321+0000", 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019243871999999999 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.020330+0000", 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2369999999999998e-06 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:39.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.020369+0000", 2024-09-16T11:43:39.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9233999999999997e-05 2024-09-16T11:43:39.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:39.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:39.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:39.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.020406+0000", 2024-09-16T11:43:39.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6649000000000001e-05 2024-09-16T11:43:39.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.223 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:39.223 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:39.223 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:39.224 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:43:39.225 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:43:39.225 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:43:39.226 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:43:39.226 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:43:39.226 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:39.223+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:39.227 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:43:39.227 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:43:39.227 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:43:39.227 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:43:39.228 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:43:39.228 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:43:39.230 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:43:39.230 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:43:39.246 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:43:39.246 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:43:39.261 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:43:39.261 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:43:39.325 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:39.324+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:39.364 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:39.365 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:39.372 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:43:39.425 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:39.423+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:39.509 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:39.509 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:39.517 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:43:39.561 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:39.560+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:39.617 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:39.617 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:39.625 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:43:39.666 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:39.665+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:39.727 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:39.725+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:39.762 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:39.763 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:39.771 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:43:39.795 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T11:43:39.851 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:39.849+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:39.861 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone3 from 0xe0cd068 (overlap 0xe0cd068) to 0xca521f7 2024-09-16T11:43:39.914 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:39.914 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:39.923 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:43:39.967 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:39.966+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:40.023 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:40.023 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:40.029 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:40.026+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:40.032 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:43:40.035 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-16T11:43:40.039 INFO:teuthology.orchestra.run.smithi070.stdout:514 trunc from 0xe0cd068 to 0xada5678 2024-09-16T11:43:40.053 INFO:teuthology.orchestra.run.smithi070.stdout:516 write 0x19673a3 thru 0x1975fb4 (0xec12 bytes) 2024-09-16T11:43:40.068 INFO:teuthology.orchestra.run.smithi070.stdout:517 trunc from 0xada5678 to 0x32516f3 2024-09-16T11:43:40.088 INFO:teuthology.orchestra.run.smithi070.stdout:518 punch from 0x1040e69 to 0x104eb5b, (0xdcf2 bytes) 2024-09-16T11:43:40.102 INFO:teuthology.orchestra.run.smithi070.stdout:519 write 0xb6b4313 thru 0xb6bc184 (0x7e72 bytes) 2024-09-16T11:43:40.106 INFO:teuthology.orchestra.run.smithi070.stdout:520 write 0xa8ba6fa thru 0xa8c6902 (0xc209 bytes) 2024-09-16T11:43:40.109 INFO:teuthology.orchestra.run.smithi070.stdout:522 read 0x45e8fdc thru 0x45f44df (0xb504 bytes) 2024-09-16T11:43:40.110 INFO:teuthology.orchestra.run.smithi070.stdout:523 write 0x416f065 thru 0x416fe3c (0xdd8 bytes) 2024-09-16T11:43:40.123 INFO:teuthology.orchestra.run.smithi070.stdout:524 trunc from 0xb6bc185 to 0x23b155e 2024-09-16T11:43:40.130 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:40.127+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:40.149 INFO:teuthology.orchestra.run.smithi070.stdout:525 trunc from 0x23b155e to 0x423ba31 2024-09-16T11:43:40.157 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:40.158 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:40.161 INFO:teuthology.orchestra.run.smithi070.stdout:526 write 0x64c8da6 thru 0x64cb2ad (0x2508 bytes) 2024-09-16T11:43:40.164 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:40.168 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:43:40.172 INFO:teuthology.orchestra.run.smithi070.stdout:528 writesame 0x922fd4d thru 0x9231c3c data_size 0x50(0x1ef0 bytes) 2024-09-16T11:43:40.172 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:40.172 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:40.172 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:40.172 INFO:teuthology.orchestra.run.smithi070.stdout:533 punch from 0x7b575ce to 0x7b59df1, (0x2823 bytes) 2024-09-16T11:43:40.176 INFO:teuthology.orchestra.run.smithi070.stdout:534 write 0x948f77d thru 0x9495a2e (0x62b2 bytes) 2024-09-16T11:43:40.180 INFO:teuthology.orchestra.run.smithi070.stdout:535 write 0x47d38c0 thru 0x47d7cb9 (0x43fa bytes) 2024-09-16T11:43:40.183 INFO:teuthology.orchestra.run.smithi070.stdout:536 write 0x7a261ab thru 0x7a2cc72 (0x6ac8 bytes) 2024-09-16T11:43:40.186 INFO:teuthology.orchestra.run.smithi070.stdout:537 punch from 0x786ce9b to 0x7873e8a, (0x6fef bytes) 2024-09-16T11:43:40.197 INFO:teuthology.orchestra.run.smithi070.stdout:538 trunc from 0x9495a2f to 0x2b5cea 2024-09-16T11:43:40.226 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:40.227 INFO:teuthology.orchestra.run.smithi070.stdout:540 read 0xdfa11 thru 0xdfecc (0x4bc bytes) 2024-09-16T11:43:40.230 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:40.228+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:40.242 INFO:teuthology.orchestra.run.smithi070.stdout:545 write 0x990a5e5 thru 0x990f77d (0x5199 bytes) 2024-09-16T11:43:40.245 INFO:teuthology.orchestra.run.smithi070.stdout:546 writesame 0x798f9f0 thru 0x7993ffb data_size 0x2(0x460c bytes) 2024-09-16T11:43:40.251 INFO:teuthology.orchestra.run.smithi070.stdout:548 read 0x5fbd4cb thru 0x5fca248 (0xcd7e bytes) 2024-09-16T11:43:40.251 INFO:teuthology.orchestra.run.smithi070.stdout:552 clone 5 order 23 su 4194304 sc 10 2024-09-16T11:43:40.275 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:43:40.275 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:43:40.275 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-16T11:43:40.275 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-16T11:43:40.283 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:40.283 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:40.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:43:40.333 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:40.331+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:40.432 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:40.433 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:40.433 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:40.431+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:40.441 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:43:40.534 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:40.531+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:40.539 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:40.539 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:40.542 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:43:40.542 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:43:40.542 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.19179 -> 3.19179 (expected baseline 3.13748) 2024-09-16T11:43:40.542 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:43:40.542 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 18 -> 18 pgs (1.6 -> 1.6 * mean) 2024-09-16T11:43:40.542 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:40.542 INFO:teuthology.orchestra.run.smithi070.stdout:oload 1000 2024-09-16T11:43:40.542 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 0.05 2024-09-16T11:43:40.542 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:43:40.543 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0032 2024-09-16T11:43:40.543 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0318 2024-09-16T11:43:40.543 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:43:40.548 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:43:40.554 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T11:43:40.555 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-16T11:43:40.635 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:40.633+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:40.697 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:40.697 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:40.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:43:40.750 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:40.748+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:40.810 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:40.810 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:40.819 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:43:40.836 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:40.835+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:40.874 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone4 from 0x990f77e (overlap 0x2b5cea) to 0x2706d2 2024-09-16T11:43:40.876 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:43:40.876 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:43:40.876 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.19179 -> 3.19179 (expected baseline 3.13748) 2024-09-16T11:43:40.876 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:43:40.877 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 18 -> 18 pgs (1.6 -> 1.6 * mean) 2024-09-16T11:43:40.877 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:40.877 INFO:teuthology.orchestra.run.smithi070.stdout:oload 110 2024-09-16T11:43:40.877 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 3 2024-09-16T11:43:40.877 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:43:40.877 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0032 2024-09-16T11:43:40.877 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0035 2024-09-16T11:43:40.877 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:43:40.891 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T11:43:40.891 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-16T11:43:40.937 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:40.936+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:40.968 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:40.969 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-16T11:43:40.979 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:43:41.039 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:41.037+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:41.139 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:41.137+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:41.144 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:41.144 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:41.153 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:43:41.230 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:43:41.230 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:43:41.230 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.19179 -> 3.19179 (expected baseline 3.13748) 2024-09-16T11:43:41.230 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:43:41.230 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 18 -> 18 pgs (1.6 -> 1.6 * mean) 2024-09-16T11:43:41.230 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:41.230 INFO:teuthology.orchestra.run.smithi070.stdout:oload 110 2024-09-16T11:43:41.230 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 3 2024-09-16T11:43:41.230 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:43:41.230 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0032 2024-09-16T11:43:41.230 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0035 2024-09-16T11:43:41.231 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:43:41.239 INFO:teuthology.orchestra.run.smithi070.stdout:553 read 0x454cd0 thru 0x45847f (0x37b0 bytes) 2024-09-16T11:43:41.240 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:41.238+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:41.240 INFO:teuthology.orchestra.run.smithi070.stdout:554 write 0x468a2e2 thru 0x469775e (0xd47d bytes) 2024-09-16T11:43:41.243 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T11:43:41.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-16T11:43:41.257 INFO:teuthology.orchestra.run.smithi070.stdout:555 write 0xe18961c thru 0xe1931ca (0x9baf bytes) 2024-09-16T11:43:41.261 INFO:teuthology.orchestra.run.smithi070.stdout:559 read 0x2e77a6a thru 0x2e801d0 (0x8767 bytes) 2024-09-16T11:43:41.261 INFO:teuthology.orchestra.run.smithi070.stdout:560 read 0x8a6c70c thru 0x8a7aaf9 (0xe3ee bytes) 2024-09-16T11:43:41.261 INFO:teuthology.orchestra.run.smithi070.stdout:561 writesame 0xb9d8248 thru 0xb9dcae4 data_size 0x1(0x489d bytes) 2024-09-16T11:43:41.271 INFO:teuthology.orchestra.run.smithi070.stdout:563 write 0x15f0a93 thru 0x15fc7fc (0xbd6a bytes) 2024-09-16T11:43:41.272 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:43:41.274 INFO:teuthology.orchestra.run.smithi070.stdout:564 writesame 0x79ed204 thru 0x79fb410 data_size 0x7(0xe20d bytes) 2024-09-16T11:43:41.281 INFO:teuthology.orchestra.run.smithi070.stdout:565 write 0x12593bb thru 0x125d7d7 (0x441d bytes) 2024-09-16T11:43:41.285 INFO:teuthology.orchestra.run.smithi070.stdout:566 read 0x3e4c249 thru 0x3e5b67f (0xf437 bytes) 2024-09-16T11:43:41.286 INFO:teuthology.orchestra.run.smithi070.stdout:567 read 0x5f26e0a thru 0x5f2f2bc (0x84b3 bytes) 2024-09-16T11:43:41.286 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:41.286 INFO:teuthology.orchestra.run.smithi070.stdout:569 write 0x7982e1e thru 0x798ce73 (0xa056 bytes) 2024-09-16T11:43:41.288 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:41.288 INFO:teuthology.orchestra.run.smithi070.stdout:572 punch from 0xdfd8c4b to 0xdfe1630, (0x89e5 bytes) 2024-09-16T11:43:41.288 INFO:teuthology.orchestra.run.smithi070.stdout:573 clone 6 order 22 su 262144 sc 14 2024-09-16T11:43:41.302 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:41.302 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:41.311 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:43:41.353 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:41.351+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:41.394 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:43:41.394 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:43:41.410 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:43:41.410 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:43:41.425 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:43:41.425 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:43:41.442 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:41.440+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:41.445 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:43:41.445 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:43:41.464 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:43:41.464 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:43:41.465 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:41.465 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:41.465 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:41.465 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:41.475 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:43:41.480 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:43:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:43:41.498 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:43:41.499 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:43:41.518 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:43:41.518 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:43:41.532 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone5 from 0xe1931cb (overlap 0x990f77e) to 0x7a72c64 2024-09-16T11:43:41.542 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:41.541+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:41.561 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:43:41.561 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:43:41.561 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.19179 -> 3.19179 (expected baseline 3.13748) 2024-09-16T11:43:41.561 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:43:41.561 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 18 -> 18 pgs (1.6 -> 1.6 * mean) 2024-09-16T11:43:41.561 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:41.561 INFO:teuthology.orchestra.run.smithi070.stdout:oload 1000 2024-09-16T11:43:41.561 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 3 2024-09-16T11:43:41.561 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:43:41.561 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0032 2024-09-16T11:43:41.561 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0320 2024-09-16T11:43:41.562 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:43:41.575 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T11:43:41.575 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-16T11:43:41.593 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:41.594 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:41.594 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:41.594 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:41.594 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:41.603 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:43:41.643 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:41.641+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:41.707 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-16T11:43:41.716 INFO:teuthology.orchestra.run.smithi070.stdout:574 write 0x45294ec thru 0x453269a (0x91af bytes) 2024-09-16T11:43:41.724 INFO:teuthology.orchestra.run.smithi070.stdout:575 punch from 0x2abea9e to 0x2acd9d1, (0xef33 bytes) 2024-09-16T11:43:41.724 INFO:teuthology.orchestra.run.smithi070.stdout:576 read 0xb0208b0 thru 0xb02479a (0x3eeb bytes) 2024-09-16T11:43:41.724 INFO:teuthology.orchestra.run.smithi070.stdout:577 read 0x621fc03 thru 0x622339e (0x379c bytes) 2024-09-16T11:43:41.724 INFO:teuthology.orchestra.run.smithi070.stdout:578 write 0xc2d8545 thru 0xc2e6a1f (0xe4db bytes) 2024-09-16T11:43:41.733 INFO:teuthology.orchestra.run.smithi070.stdout:579 trunc from 0xe1931cb to 0x93d3f46 2024-09-16T11:43:41.743 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:41.742+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:41.765 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2154 4.11 4:8cd0e658:::rbd_data.11c17b8dc4e5.0000000000000002:5 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.865479+0000", 2024-09-16T11:43:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8965691489999998, 2024-09-16T11:43:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016073390999999999, 2024-09-16T11:43:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2154 2024-09-16T11:43:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865479+0000", 2024-09-16T11:43:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865479+0000", 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865479+0000", 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.05e-07 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865480+0000", 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7499999999999998e-07 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865483+0000", 2024-09-16T11:43:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8650000000000001e-06 2024-09-16T11:43:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865485+0000", 2024-09-16T11:43:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8899999999999999e-06 2024-09-16T11:43:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.881399+0000", 2024-09-16T11:43:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.015914456 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.881471+0000", 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1398999999999999e-05 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.881552+0000", 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1600999999999993e-05 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2155 4.11 4:8cd0e658:::rbd_data.11c17b8dc4e5.0000000000000002:5 [read 1966080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.865487+0000", 2024-09-16T11:43:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8965612200000002, 2024-09-16T11:43:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016260223000000001, 2024-09-16T11:43:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2155 2024-09-16T11:43:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865487+0000", 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865487+0000", 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865487+0000", 2024-09-16T11:43:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8999999999999998e-07 2024-09-16T11:43:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865490+0000", 2024-09-16T11:43:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1020000000000001e-06 2024-09-16T11:43:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865500+0000", 2024-09-16T11:43:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4550000000000001e-06 2024-09-16T11:43:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865503+0000", 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.948e-06 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.881588+0000", 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016085827 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.881664+0000", 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5624000000000006e-05 2024-09-16T11:43:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.881747+0000", 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2977000000000002e-05 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2156 4.11 4:8cd0e658:::rbd_data.11c17b8dc4e5.0000000000000002:5 [read 2162688~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.865505+0000", 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8965430329999999, 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016418377000000001, 2024-09-16T11:43:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2156 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865505+0000", 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865505+0000", 2024-09-16T11:43:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865505+0000", 2024-09-16T11:43:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.17e-07 2024-09-16T11:43:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865511+0000", 2024-09-16T11:43:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6110000000000001e-06 2024-09-16T11:43:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865516+0000", 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.31e-06 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865518+0000", 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9460000000000001e-06 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.881777+0000", 2024-09-16T11:43:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016259010000000001 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.881839+0000", 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1588999999999996e-05 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.881923+0000", 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4693999999999999e-05 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2157 4.11 4:8cd0e658:::rbd_data.11c17b8dc4e5.0000000000000002:5 [read 2359296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.865521+0000", 2024-09-16T11:43:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.896527356, 2024-09-16T11:43:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016557974999999999, 2024-09-16T11:43:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2157 2024-09-16T11:43:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865521+0000", 2024-09-16T11:43:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865521+0000", 2024-09-16T11:43:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865521+0000", 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.29e-07 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865522+0000", 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1100000000000004e-07 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865525+0000", 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3419999999999999e-06 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865527+0000", 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8160000000000001e-06 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.881958+0000", 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016430995 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.882017+0000", 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8754e-05 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.882079+0000", 2024-09-16T11:43:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1927999999999994e-05 2024-09-16T11:43:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2158 4.11 4:8cd0e658:::rbd_data.11c17b8dc4e5.0000000000000002:5 [read 2605056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.865529+0000", 2024-09-16T11:43:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8965189580000001, 2024-09-16T11:43:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016763697000000001, 2024-09-16T11:43:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2158 2024-09-16T11:43:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865529+0000", 2024-09-16T11:43:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865529+0000", 2024-09-16T11:43:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865529+0000", 2024-09-16T11:43:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0699999999999999e-07 2024-09-16T11:43:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865530+0000", 2024-09-16T11:43:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8100000000000001e-07 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865533+0000", 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7609999999999999e-06 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865535+0000", 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7600000000000001e-06 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.882119+0000", 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016584365 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.882196+0000", 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6649000000000004e-05 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.882293+0000", 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7073999999999999e-05 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2169 4.1d 4:b8712c40:::rbd_data.11c17b8dc4e5.0000000000000003:5 [read 2588672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.865642+0000", 2024-09-16T11:43:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.896405927, 2024-09-16T11:43:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016019367, 2024-09-16T11:43:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2169 2024-09-16T11:43:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865642+0000", 2024-09-16T11:43:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865642+0000", 2024-09-16T11:43:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865643+0000", 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9999999999999998e-07 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865644+0000", 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6530000000000001e-06 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865650+0000", 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.865653+0000", 2024-09-16T11:43:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2660000000000002e-06 2024-09-16T11:43:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.881572+0000", 2024-09-16T11:43:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.015919355 2024-09-16T11:43:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.881613+0000", 2024-09-16T11:43:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0166000000000001e-05 2024-09-16T11:43:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.881661+0000", 2024-09-16T11:43:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8791000000000001e-05 2024-09-16T11:43:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2197 4.11 4:8cd0e658:::rbd_data.11c17b8dc4e5.0000000000000002:5 [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.866857+0000", 2024-09-16T11:43:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8951906859999998, 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016106754000000001, 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2197 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866857+0000", 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866857+0000", 2024-09-16T11:43:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866857+0000", 2024-09-16T11:43:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1e-07 2024-09-16T11:43:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866858+0000", 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3900000000000003e-07 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866861+0000", 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8679999999999999e-06 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866863+0000", 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9570000000000001e-06 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.882796+0000", 2024-09-16T11:43:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.015932595000000001 2024-09-16T11:43:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.882876+0000", 2024-09-16T11:43:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0078000000000001e-05 2024-09-16T11:43:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.882964+0000", 2024-09-16T11:43:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8107000000000002e-05 2024-09-16T11:43:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2198 4.11 4:8cd0e658:::rbd_data.11c17b8dc4e5.0000000000000002:5 [read 1081344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.866865+0000", 2024-09-16T11:43:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8951829130000002, 2024-09-16T11:43:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016312640999999999, 2024-09-16T11:43:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2198 2024-09-16T11:43:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866865+0000", 2024-09-16T11:43:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866865+0000", 2024-09-16T11:43:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866865+0000", 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.05e-07 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866866+0000", 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8400000000000003e-07 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866869+0000", 2024-09-16T11:43:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:43:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866872+0000", 2024-09-16T11:43:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9919999999999999e-06 2024-09-16T11:43:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.882997+0000", 2024-09-16T11:43:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016125029999999999 2024-09-16T11:43:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883070+0000", 2024-09-16T11:43:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2596999999999996e-05 2024-09-16T11:43:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883178+0000", 2024-09-16T11:43:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000107964 2024-09-16T11:43:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2199 4.11 4:8cd0e658:::rbd_data.11c17b8dc4e5.0000000000000002:5 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.866874+0000", 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8951738840000001, 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016493279999999999, 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2199 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866874+0000", 2024-09-16T11:43:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866874+0000", 2024-09-16T11:43:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866874+0000", 2024-09-16T11:43:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0600000000000001e-07 2024-09-16T11:43:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866875+0000", 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6099999999999999e-07 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866878+0000", 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8250000000000001e-06 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866880+0000", 2024-09-16T11:43:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.885e-06 2024-09-16T11:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883208+0000", 2024-09-16T11:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016327901999999998 2024-09-16T11:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883284+0000", 2024-09-16T11:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5555000000000004e-05 2024-09-16T11:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883367+0000", 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3845999999999996e-05 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2200 4.11 4:8cd0e658:::rbd_data.11c17b8dc4e5.0000000000000002:5 [read 1720320~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.866902+0000", 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.895145495, 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016624178, 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2200 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866902+0000", 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866902+0000", 2024-09-16T11:43:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866903+0000", 2024-09-16T11:43:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7899999999999999e-07 2024-09-16T11:43:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866904+0000", 2024-09-16T11:43:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T11:43:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866910+0000", 2024-09-16T11:43:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4110000000000004e-06 2024-09-16T11:43:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866913+0000", 2024-09-16T11:43:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5279999999999999e-06 2024-09-16T11:43:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883394+0000", 2024-09-16T11:43:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016480558999999999 2024-09-16T11:43:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883448+0000", 2024-09-16T11:43:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4376999999999998e-05 2024-09-16T11:43:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883527+0000", 2024-09-16T11:43:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8283999999999998e-05 2024-09-16T11:43:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2201 4.11 4:8cd0e658:::rbd_data.11c17b8dc4e5.0000000000000002:5 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.866917+0000", 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.895130687, 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016757824000000001, 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2201 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866917+0000", 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866917+0000", 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866918+0000", 2024-09-16T11:43:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4900000000000001e-07 2024-09-16T11:43:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866920+0000", 2024-09-16T11:43:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:43:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866925+0000", 2024-09-16T11:43:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8200000000000002e-06 2024-09-16T11:43:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866929+0000", 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3680000000000002e-06 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883561+0000", 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016632273 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883613+0000", 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2335e-05 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883675+0000", 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1744999999999999e-05 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2202 4.11 4:8cd0e658:::rbd_data.11c17b8dc4e5.0000000000000002:5 [read 2113536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.866932+0000", 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8951156949999999, 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016922671, 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2202 2024-09-16T11:43:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866932+0000", 2024-09-16T11:43:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866932+0000", 2024-09-16T11:43:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866933+0000", 2024-09-16T11:43:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:43:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866934+0000", 2024-09-16T11:43:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7579999999999999e-06 2024-09-16T11:43:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866941+0000", 2024-09-16T11:43:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4130000000000003e-06 2024-09-16T11:43:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866944+0000", 2024-09-16T11:43:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3569999999999998e-06 2024-09-16T11:43:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883697+0000", 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016753209000000002 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883766+0000", 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9146000000000005e-05 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883855+0000", 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8437999999999996e-05 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2203 4.11 4:8cd0e658:::rbd_data.11c17b8dc4e5.0000000000000002:5 [read 2555904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.866948+0000", 2024-09-16T11:43:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.895099976, 2024-09-16T11:43:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.017030541, 2024-09-16T11:43:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2203 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866948+0000", 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866948+0000", 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866948+0000", 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.96e-07 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866950+0000", 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9130000000000001e-06 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866956+0000", 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3720000000000004e-06 2024-09-16T11:43:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.866959+0000", 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4149999999999999e-06 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883875+0000", 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016915955 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883927+0000", 2024-09-16T11:43:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1875000000000001e-05 2024-09-16T11:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883979+0000", 2024-09-16T11:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1615e-05 2024-09-16T11:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2211 4.1d 4:b8712c40:::rbd_data.11c17b8dc4e5.0000000000000003:5 [read 1900544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.867053+0000", 2024-09-16T11:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8949946510000002, 2024-09-16T11:43:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016161366999999999, 2024-09-16T11:43:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2211 2024-09-16T11:43:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867053+0000", 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867053+0000", 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867054+0000", 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867055+0000", 2024-09-16T11:43:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.685e-06 2024-09-16T11:43:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867061+0000", 2024-09-16T11:43:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4770000000000002e-06 2024-09-16T11:43:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867064+0000", 2024-09-16T11:43:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:43:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.882957+0000", 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.015892944999999999 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883082+0000", 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012505799999999999 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883215+0000", 2024-09-16T11:43:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013254 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2212 4.1d 4:b8712c40:::rbd_data.11c17b8dc4e5.0000000000000003:5 [read 2097152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.867068+0000", 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8949801490000002, 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016401183999999999, 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2212 2024-09-16T11:43:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867068+0000", 2024-09-16T11:43:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867068+0000", 2024-09-16T11:43:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867068+0000", 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9000000000000002e-07 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867070+0000", 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6789999999999999e-06 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867075+0000", 2024-09-16T11:43:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5199999999999997e-06 2024-09-16T11:43:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867079+0000", 2024-09-16T11:43:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3419999999999999e-06 2024-09-16T11:43:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883244+0000", 2024-09-16T11:43:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016164920999999999 2024-09-16T11:43:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883348+0000", 2024-09-16T11:43:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104678 2024-09-16T11:43:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883469+0000", 2024-09-16T11:43:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000120654 2024-09-16T11:43:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2213 4.1d 4:b8712c40:::rbd_data.11c17b8dc4e5.0000000000000003:5 [read 2342912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.867082+0000", 2024-09-16T11:43:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8949657950000001, 2024-09-16T11:43:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016565514999999999, 2024-09-16T11:43:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2213 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867082+0000", 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867082+0000", 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867082+0000", 2024-09-16T11:43:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:43:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867084+0000", 2024-09-16T11:43:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.57e-06 2024-09-16T11:43:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867089+0000", 2024-09-16T11:43:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:43:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867106+0000", 2024-09-16T11:43:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6697e-05 2024-09-16T11:43:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883492+0000", 2024-09-16T11:43:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016385733999999999 2024-09-16T11:43:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883566+0000", 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4004000000000005e-05 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883648+0000", 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1773999999999993e-05 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2214 4.1d 4:b8712c40:::rbd_data.11c17b8dc4e5.0000000000000003:5 [read 2539520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.867110+0000", 2024-09-16T11:43:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8949375829999999, 2024-09-16T11:43:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016771334999999998, 2024-09-16T11:43:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2214 2024-09-16T11:43:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867110+0000", 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867110+0000", 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867111+0000", 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0999999999999999e-07 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867112+0000", 2024-09-16T11:43:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.387e-06 2024-09-16T11:43:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867116+0000", 2024-09-16T11:43:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4970000000000001e-06 2024-09-16T11:43:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867118+0000", 2024-09-16T11:43:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.65e-06 2024-09-16T11:43:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883688+0000", 2024-09-16T11:43:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016569943 2024-09-16T11:43:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883789+0000", 2024-09-16T11:43:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000101036 2024-09-16T11:43:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883882+0000", 2024-09-16T11:43:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2411999999999995e-05 2024-09-16T11:43:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2215 4.19 4:9f0c94cb:::rbd_object_map.11c130c94979.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.867474+0000", 2024-09-16T11:43:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8945737199999999, 2024-09-16T11:43:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.019978115000000001, 2024-09-16T11:43:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2215 2024-09-16T11:43:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867474+0000", 2024-09-16T11:43:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867474+0000", 2024-09-16T11:43:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867475+0000", 2024-09-16T11:43:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2999999999999999e-07 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867479+0000", 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9890000000000003e-06 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867489+0000", 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0101e-05 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867494+0000", 2024-09-16T11:43:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9740000000000001e-06 2024-09-16T11:43:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883895+0000", 2024-09-16T11:43:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016400676999999999 2024-09-16T11:43:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.883971+0000", 2024-09-16T11:43:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6131999999999995e-05 2024-09-16T11:43:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2", 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.884321+0000", 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000349842 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.886336+0000", 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002015691 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.886585+0000", 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024833 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.886609+0000", 2024-09-16T11:43:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4365000000000001e-05 2024-09-16T11:43:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.887452+0000", 2024-09-16T11:43:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00084318399999999997 2024-09-16T11:43:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2216 4.19 4:9f0c94cb:::rbd_object_map.11c130c94979.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.867643+0000", 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8944050649999999, 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.019887453999999999, 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2216 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867643+0000", 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867643+0000", 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867644+0000", 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2399999999999997e-07 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867654+0000", 2024-09-16T11:43:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0400999999999999e-05 2024-09-16T11:43:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867663+0000", 2024-09-16T11:43:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5900000000000008e-06 2024-09-16T11:43:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.867667+0000", 2024-09-16T11:43:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6029999999999998e-06 2024-09-16T11:43:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.884673+0000", 2024-09-16T11:43:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.017006248000000002 2024-09-16T11:43:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.884742+0000", 2024-09-16T11:43:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8172999999999999e-05 2024-09-16T11:43:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2", 2024-09-16T11:43:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.885013+0000", 2024-09-16T11:43:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00027092199999999999 2024-09-16T11:43:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.886578+0000", 2024-09-16T11:43:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001565863 2024-09-16T11:43:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.887460+0000", 2024-09-16T11:43:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00088204500000000003 2024-09-16T11:43:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.887497+0000", 2024-09-16T11:43:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6906999999999997e-05 2024-09-16T11:43:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.887530+0000", 2024-09-16T11:43:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2978000000000002e-05 2024-09-16T11:43:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2218 4.19 4:9f0c94cb:::rbd_object_map.11c130c94979.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.871377+0000", 2024-09-16T11:43:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 2.8906707360000001, 2024-09-16T11:43:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.017286155000000001, 2024-09-16T11:43:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2218 2024-09-16T11:43:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.871377+0000", 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.871377+0000", 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.871378+0000", 2024-09-16T11:43:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.204e-06 2024-09-16T11:43:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.871383+0000", 2024-09-16T11:43:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:43:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.871407+0000", 2024-09-16T11:43:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4090000000000001e-05 2024-09-16T11:43:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.871413+0000", 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9780000000000002e-06 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.885479+0000", 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014065190999999999 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.885712+0000", 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000233031 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2", 2024-09-16T11:43:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.886021+0000", 2024-09-16T11:43:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00030919399999999999 2024-09-16T11:43:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.888271+0000", 2024-09-16T11:43:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022500530000000001 2024-09-16T11:43:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.888471+0000", 2024-09-16T11:43:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000200001 2024-09-16T11:43:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.888625+0000", 2024-09-16T11:43:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015387899999999999 2024-09-16T11:43:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.888663+0000", 2024-09-16T11:43:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8714999999999998e-05 2024-09-16T11:43:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.862 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:41.862 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:41.862 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:41.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:43:41.864 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:43:41.865 INFO:teuthology.orchestra.run.smithi070.stdout:580 trunc from 0x93d3f46 to 0x3dab4f1 2024-09-16T11:43:41.865 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:41.865 INFO:teuthology.orchestra.run.smithi070.stdout:582 read 0x2e64f0b thru 0x2e6c1c0 (0x72b6 bytes) 2024-09-16T11:43:41.865 INFO:teuthology.orchestra.run.smithi070.stdout:584 punch from 0x1e236a8 to 0x1e25f4b, (0x28a3 bytes) 2024-09-16T11:43:41.865 INFO:teuthology.orchestra.run.smithi070.stdout:585 writesame 0x68d144b thru 0x68d70a7 data_size 0x1(0x5c5d bytes) 2024-09-16T11:43:41.866 INFO:teuthology.orchestra.run.smithi070.stdout:586 write 0x903129 thru 0x9092d9 (0x61b1 bytes) 2024-09-16T11:43:41.866 INFO:teuthology.orchestra.run.smithi070.stdout:589 trunc from 0x68d70a8 to 0x6ed2555 2024-09-16T11:43:41.866 INFO:teuthology.orchestra.run.smithi070.stdout:590 writesame 0xc49535e thru 0xc49545e data_size 0x1(0x101 bytes) 2024-09-16T11:43:41.866 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:41.866 INFO:teuthology.orchestra.run.smithi070.stdout:594 trunc from 0xc49545f to 0x4371569 2024-09-16T11:43:41.866 INFO:teuthology.orchestra.run.smithi070.stdout:595 read 0x1e65283 thru 0x1e6fd22 (0xaaa0 bytes) 2024-09-16T11:43:41.866 INFO:teuthology.orchestra.run.smithi070.stdout:597 write 0x760dffe thru 0x761c701 (0xe704 bytes) 2024-09-16T11:43:41.866 INFO:teuthology.orchestra.run.smithi070.stdout:598 read 0x4a95645 thru 0x4a9d51e (0x7eda bytes) 2024-09-16T11:43:41.866 INFO:teuthology.orchestra.run.smithi070.stdout:599 read 0xb08bb thru 0xbbbbf (0xb305 bytes) 2024-09-16T11:43:41.867 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:41.862+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:41.878 INFO:teuthology.orchestra.run.smithi070.stdout:600 write 0xd98cbb9 thru 0xd99b346 (0xe78e bytes) 2024-09-16T11:43:41.881 INFO:teuthology.orchestra.run.smithi070.stdout:601 punch from 0xaea5aac to 0xaea8749, (0x2c9d bytes) 2024-09-16T11:43:41.881 INFO:teuthology.orchestra.run.smithi070.stdout:603 write 0x9fcd50e thru 0x9fd954b (0xc03e bytes) 2024-09-16T11:43:41.884 INFO:teuthology.orchestra.run.smithi070.stdout:604 punch from 0x9840f5e to 0x984fc2f, (0xecd1 bytes) 2024-09-16T11:43:41.886 INFO:teuthology.orchestra.run.smithi070.stdout:607 trunc from 0xd99b347 to 0xbdf442c 2024-09-16T11:43:41.893 INFO:teuthology.orchestra.run.smithi070.stdout:608 writesame 0x6c8994e thru 0x6c9236c data_size 0x1(0x8a1f bytes) 2024-09-16T11:43:41.898 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:43:41.898 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:43:41.898 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.19179 -> 3.19179 (expected baseline 3.13748) 2024-09-16T11:43:41.898 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:43:41.898 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 18 -> 18 pgs (1.6 -> 1.6 * mean) 2024-09-16T11:43:41.898 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:43:41.899 INFO:teuthology.orchestra.run.smithi070.stdout:oload 110 2024-09-16T11:43:41.899 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 3 2024-09-16T11:43:41.899 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:43:41.899 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0032 2024-09-16T11:43:41.899 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0035 2024-09-16T11:43:41.899 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:43:41.924 INFO:teuthology.orchestra.run.smithi070.stdout:609 trunc from 0xbdf442c to 0xd50928 2024-09-16T11:43:41.958 INFO:teuthology.orchestra.run.smithi070.stdout:611 read 0x817e68 thru 0x826a31 (0xebca bytes) 2024-09-16T11:43:41.958 INFO:teuthology.orchestra.run.smithi070.stdout:612 read 0xa8606b thru 0xa94826 (0xe7bc bytes) 2024-09-16T11:43:41.958 INFO:teuthology.orchestra.run.smithi070.stdout:615 punch from 0xce4e57 to 0xcefe3b, (0xafe4 bytes) 2024-09-16T11:43:41.964 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:41.963+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:41.976 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:43:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:5345 4.9 4:92d2cc75:::rbd_header.11c18a2497a7:head [call rbd.set_size in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:41.974355+0000", 2024-09-16T11:43:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.00051160599999999995, 2024-09-16T11:43:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000534773, 2024-09-16T11:43:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "waiting for sub ops", 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5345 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.974355+0000", 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.974355+0000", 2024-09-16T11:43:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.974356+0000", 2024-09-16T11:43:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7800000000000001e-07 2024-09-16T11:43:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.974362+0000", 2024-09-16T11:43:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4160000000000001e-06 2024-09-16T11:43:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.974380+0000", 2024-09-16T11:43:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7095000000000001e-05 2024-09-16T11:43:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.974386+0000", 2024-09-16T11:43:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5130000000000002e-06 2024-09-16T11:43:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.974414+0000", 2024-09-16T11:43:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7925000000000001e-05 2024-09-16T11:43:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.974474+0000", 2024-09-16T11:43:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9817e-05 2024-09-16T11:43:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.974580+0000", 2024-09-16T11:43:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010581699999999999 2024-09-16T11:43:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-09-16T11:43:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-09-16T11:43:41.981 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:41.982 INFO:teuthology.orchestra.run.smithi070.stdout:617 write 0xd36b2c0 thru 0xd36d861 (0x25a2 bytes) 2024-09-16T11:43:41.982 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:41.982 INFO:teuthology.orchestra.run.smithi070.stdout:619 read 0x6e347d1 thru 0x6e3bd40 (0x7570 bytes) 2024-09-16T11:43:41.982 INFO:teuthology.orchestra.run.smithi070.stdout:620 punch from 0x9e057e6 to 0x9e059b4, (0x1ce bytes) 2024-09-16T11:43:41.982 INFO:teuthology.orchestra.run.smithi070.stdout:621 writesame 0xa0314ca thru 0xa03c219 data_size 0x1(0xad50 bytes) 2024-09-16T11:43:41.986 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:43:42.001 INFO:teuthology.orchestra.run.smithi070.stdout:624 read 0xb203c4 thru 0xb287f6 (0x8433 bytes) 2024-09-16T11:43:42.002 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.002 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.002 INFO:teuthology.orchestra.run.smithi070.stdout:628 read 0xcb1448 thru 0xcb6269 (0x4e22 bytes) 2024-09-16T11:43:42.002 INFO:teuthology.orchestra.run.smithi070.stdout:629 writesame 0x37d88c7 thru 0x37e636e data_size 0x4(0xdaa8 bytes) 2024-09-16T11:43:42.021 INFO:teuthology.orchestra.run.smithi070.stdout:631 trunc from 0xd36d862 to 0x326db0c 2024-09-16T11:43:42.035 INFO:teuthology.orchestra.run.smithi070.stdout:633 punch from 0x166e35e to 0x16796ea, (0xb38c bytes) 2024-09-16T11:43:42.035 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.035 INFO:teuthology.orchestra.run.smithi070.stdout:635 read 0x2631a97 thru 0x26352fd (0x3867 bytes) 2024-09-16T11:43:42.035 INFO:teuthology.orchestra.run.smithi070.stdout:636 punch from 0xa6ad06 to 0xa72981, (0x7c7b bytes) 2024-09-16T11:43:42.047 INFO:teuthology.orchestra.run.smithi070.stdout:638 trunc from 0x326db0c to 0xdc73946 2024-09-16T11:43:42.052 INFO:teuthology.orchestra.run.smithi070.stdout:639 write 0xbadd17b thru 0xbaeb417 (0xe29d bytes) 2024-09-16T11:43:42.056 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.056 INFO:teuthology.orchestra.run.smithi070.stdout:642 punch from 0x7ba9006 to 0x7ba9d61, (0xd5b bytes) 2024-09-16T11:43:42.056 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.056 INFO:teuthology.orchestra.run.smithi070.stdout:644 write 0xb538518 thru 0xb5462ca (0xddb3 bytes) 2024-09-16T11:43:42.059 INFO:teuthology.orchestra.run.smithi070.stdout:646 writesame 0x2aa8743 thru 0x2ab5e17 data_size 0x1(0xd6d5 bytes) 2024-09-16T11:43:42.070 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:42.069+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:42.082 INFO:teuthology.orchestra.run.smithi070.stdout:648 write 0x2339a63 thru 0x233a1af (0x74d bytes) 2024-09-16T11:43:42.084 INFO:teuthology.orchestra.run.smithi070.stdout:649 writesame 0x23d20cc thru 0x23d4b17 data_size 0x2(0x2a4c bytes) 2024-09-16T11:43:42.086 INFO:teuthology.orchestra.run.smithi070.stdout:651 punch from 0x9330c0a to 0x933eabb, (0xdeb1 bytes) 2024-09-16T11:43:42.087 INFO:teuthology.orchestra.run.smithi070.stdout:652 write 0x5aff8dc thru 0x5b0bbd8 (0xc2fd bytes) 2024-09-16T11:43:42.090 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.090 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.091 INFO:teuthology.orchestra.run.smithi070.stdout:657 write 0x7b8e904 thru 0x7b91923 (0x3020 bytes) 2024-09-16T11:43:42.093 INFO:teuthology.orchestra.run.smithi070.stdout:659 punch from 0x6770b4 to 0x678f9f, (0x1eeb bytes) 2024-09-16T11:43:42.093 INFO:teuthology.orchestra.run.smithi070.stdout:660 write 0x687cfe3 thru 0x687faec (0x2b0a bytes) 2024-09-16T11:43:42.095 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.102 INFO:teuthology.orchestra.run.smithi070.stdout:662 trunc from 0xdc73946 to 0x823b480 2024-09-16T11:43:42.112 INFO:teuthology.orchestra.run.smithi070.stdout:664 writesame 0x2f511d9 thru 0x2f6068b data_size 0x15(0xf4b3 bytes) 2024-09-16T11:43:42.114 INFO:teuthology.orchestra.run.smithi070.stdout:665 punch from 0x162de05 to 0x163db3b, (0xfd36 bytes) 2024-09-16T11:43:42.115 INFO:teuthology.orchestra.run.smithi070.stdout:666 punch from 0x434919a to 0x4356c45, (0xdaab bytes) 2024-09-16T11:43:42.115 INFO:teuthology.orchestra.run.smithi070.stdout:667 read 0x2abd38d thru 0x2acc387 (0xeffb bytes) 2024-09-16T11:43:42.115 INFO:teuthology.orchestra.run.smithi070.stdout:668 read 0x7a7b4dd thru 0x7a7d848 (0x236c bytes) 2024-09-16T11:43:42.115 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.115 INFO:teuthology.orchestra.run.smithi070.stdout:670 write 0x768d9df thru 0x7694eec (0x750e bytes) 2024-09-16T11:43:42.118 INFO:teuthology.orchestra.run.smithi070.stdout:671 write 0x4cf81ff thru 0x4cf87ea (0x5ec bytes) 2024-09-16T11:43:42.119 INFO:teuthology.orchestra.run.smithi070.stdout:673 punch from 0x200439d to 0x20086ad, (0x4310 bytes) 2024-09-16T11:43:42.119 INFO:teuthology.orchestra.run.smithi070.stdout:675 write 0x73b84d3 thru 0x73c6a27 (0xe555 bytes) 2024-09-16T11:43:42.126 INFO:teuthology.orchestra.run.smithi070.stdout:676 trunc from 0x823b480 to 0x45f9b7e 2024-09-16T11:43:42.133 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:42.133 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:42.133 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:42.134 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:42.134 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:42.137 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.137 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.139 INFO:teuthology.orchestra.run.smithi070.stdout:680 trunc from 0x45f9b7e to 0x2a7c7a7 2024-09-16T11:43:42.142 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:43:42.155 INFO:teuthology.orchestra.run.smithi070.stdout:682 punch from 0x1309fd6 to 0x1310b41, (0x6b6b bytes) 2024-09-16T11:43:42.155 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.155 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.155 INFO:teuthology.orchestra.run.smithi070.stdout:687 read 0x199feb2 thru 0x19a2927 (0x2a76 bytes) 2024-09-16T11:43:42.155 INFO:teuthology.orchestra.run.smithi070.stdout:688 punch from 0x1bfcb88 to 0x1bfd815, (0xc8d bytes) 2024-09-16T11:43:42.155 INFO:teuthology.orchestra.run.smithi070.stdout:689 writesame 0xd0905e thru 0xd131f3 data_size 0x4a(0xa196 bytes) 2024-09-16T11:43:42.156 INFO:teuthology.orchestra.run.smithi070.stdout:691 trunc from 0x2a7c7a7 to 0x211fd1e 2024-09-16T11:43:42.166 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:42.164+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:42.168 INFO:teuthology.orchestra.run.smithi070.stdout:692 punch from 0x1cbf8df to 0x1cc359a, (0x3cbb bytes) 2024-09-16T11:43:42.169 INFO:teuthology.orchestra.run.smithi070.stdout:696 punch from 0x83453e to 0x839e1d, (0x58df bytes) 2024-09-16T11:43:42.169 INFO:teuthology.orchestra.run.smithi070.stdout:697 punch from 0x9abf11 to 0x9b383a, (0x7929 bytes) 2024-09-16T11:43:42.169 INFO:teuthology.orchestra.run.smithi070.stdout:698 read 0x404f28 thru 0x4120f0 (0xd1c9 bytes) 2024-09-16T11:43:42.169 INFO:teuthology.orchestra.run.smithi070.stdout:699 punch from 0x11a080 to 0x11c4fc, (0x247c bytes) 2024-09-16T11:43:42.189 INFO:teuthology.orchestra.run.smithi070.stdout:700 write 0xe8ebe7d thru 0xe8f6fd3 (0xb157 bytes) 2024-09-16T11:43:42.192 INFO:teuthology.orchestra.run.smithi070.stdout:701 read 0x91bbed2 thru 0x91c6802 (0xa931 bytes) 2024-09-16T11:43:42.193 INFO:teuthology.orchestra.run.smithi070.stdout:704 writesame 0xcc8e22 thru 0xccf1e2 data_size 0x1(0x63c1 bytes) 2024-09-16T11:43:42.221 INFO:teuthology.orchestra.run.smithi070.stdout:705 trunc from 0xe8f6fd4 to 0x574ad3 2024-09-16T11:43:42.251 INFO:teuthology.orchestra.run.smithi070.stdout:706 writesame 0x9b4574c thru 0x9b4bfd7 data_size 0x2(0x688c bytes) 2024-09-16T11:43:42.251 INFO:teuthology.orchestra.run.smithi070.stdout:707 write 0x8994fc3 thru 0x899ae3a (0x5e78 bytes) 2024-09-16T11:43:42.254 INFO:teuthology.orchestra.run.smithi070.stdout:708 punch from 0x276fd55 to 0x277b7e3, (0xba8e bytes) 2024-09-16T11:43:42.266 INFO:teuthology.orchestra.run.smithi070.stdout:710 writesame 0xd79c575 thru 0xd7a65aa data_size 0x1(0xa036 bytes) 2024-09-16T11:43:42.266 INFO:teuthology.orchestra.run.smithi070.stdout:712 punch from 0xc2eab04 to 0xc2f28dc, (0x7dd8 bytes) 2024-09-16T11:43:42.266 INFO:teuthology.orchestra.run.smithi070.stdout:713 write 0xa9bfc64 thru 0xa9cd761 (0xdafe bytes) 2024-09-16T11:43:42.267 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:42.265+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:42.269 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.269 INFO:teuthology.orchestra.run.smithi070.stdout:716 read 0x3164933 thru 0x316df2a (0x95f8 bytes) 2024-09-16T11:43:42.269 INFO:teuthology.orchestra.run.smithi070.stdout:717 read 0x8f24f98 thru 0x8f31438 (0xc4a1 bytes) 2024-09-16T11:43:42.270 INFO:teuthology.orchestra.run.smithi070.stdout:718 read 0xb37b7f3 thru 0xb380c4a (0x5458 bytes) 2024-09-16T11:43:42.270 INFO:teuthology.orchestra.run.smithi070.stdout:719 writesame 0xd73e5b0 thru 0xd7491b5 data_size 0x61(0xac06 bytes) 2024-09-16T11:43:42.270 INFO:teuthology.orchestra.run.smithi070.stdout:720 write 0x9edbce7 thru 0x9ee117a (0x5494 bytes) 2024-09-16T11:43:42.273 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.273 INFO:teuthology.orchestra.run.smithi070.stdout:722 read 0x2b72246 thru 0x2b81941 (0xf6fc bytes) 2024-09-16T11:43:42.275 INFO:teuthology.orchestra.run.smithi070.stdout:724 trunc from 0xd7a65ab to 0xc0c3bf6 2024-09-16T11:43:42.284 INFO:teuthology.orchestra.run.smithi070.stdout:725 read 0x5ec4feb thru 0x5ec970f (0x4725 bytes) 2024-09-16T11:43:42.284 INFO:teuthology.orchestra.run.smithi070.stdout:726 write 0x55b95e5 thru 0x55bc575 (0x2f91 bytes) 2024-09-16T11:43:42.287 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.290 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:42.290 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:43:42.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:43:42.290 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:43:42.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.290 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:905 4.1a 4:5b7d482a:::rbd_data.11c17b8dc4e5.0000000000000008:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:42.290 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.071040+0000", 2024-09-16T11:43:42.290 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.2163229009999998, 2024-09-16T11:43:42.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.017484133999999998, 2024-09-16T11:43:42.290 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.290 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 905 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071040+0000", 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071040+0000", 2024-09-16T11:43:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071041+0000", 2024-09-16T11:43:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9299999999999998e-07 2024-09-16T11:43:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071044+0000", 2024-09-16T11:43:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:43:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071053+0000", 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9940000000000006e-06 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071057+0000", 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0990000000000001e-06 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071087+0000", 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9311000000000001e-05 2024-09-16T11:43:42.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.071157+0000", 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0207000000000001e-05 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.074097+0000", 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0029405410000000001 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.088459+0000", 2024-09-16T11:43:42.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014361237000000001 2024-09-16T11:43:42.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:42.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.088465+0000", 2024-09-16T11:43:42.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8369999999999999e-06 2024-09-16T11:43:42.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:42.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.088497+0000", 2024-09-16T11:43:42.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2048000000000002e-05 2024-09-16T11:43:42.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.088524+0000", 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6767e-05 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2044 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 2703360~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.861580+0000", 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4257834960000002, 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015214098000000001, 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.296 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.297 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2044 2024-09-16T11:43:42.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.297 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.861580+0000", 2024-09-16T11:43:42.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.861580+0000", 2024-09-16T11:43:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.861580+0000", 2024-09-16T11:43:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:43:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.861581+0000", 2024-09-16T11:43:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5510000000000001e-06 2024-09-16T11:43:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.861588+0000", 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.263e-06 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.861591+0000", 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2760000000000001e-06 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.876622+0000", 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015030979 2024-09-16T11:43:42.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.876703+0000", 2024-09-16T11:43:42.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1105000000000004e-05 2024-09-16T11:43:42.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.876794+0000", 2024-09-16T11:43:42.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0572e-05 2024-09-16T11:43:42.300 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.300 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.300 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2096 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 2195456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864406+0000", 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4229569839999998, 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015209791, 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2096 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864406+0000", 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864406+0000", 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864406+0000", 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:43:42.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864408+0000", 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5740000000000001e-06 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864413+0000", 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864417+0000", 2024-09-16T11:43:42.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2040000000000002e-06 2024-09-16T11:43:42.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879435+0000", 2024-09-16T11:43:42.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015018769 2024-09-16T11:43:42.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879523+0000", 2024-09-16T11:43:42.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7516999999999996e-05 2024-09-16T11:43:42.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879616+0000", 2024-09-16T11:43:42.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3005999999999997e-05 2024-09-16T11:43:42.305 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.305 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.305 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.305 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2097 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 2097152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.305 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864447+0000", 2024-09-16T11:43:42.305 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4229164989999998, 2024-09-16T11:43:42.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015276454, 2024-09-16T11:43:42.305 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2097 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864447+0000", 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864447+0000", 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864447+0000", 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.75e-07 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864449+0000", 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0360000000000001e-06 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864456+0000", 2024-09-16T11:43:42.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3769999999999998e-06 2024-09-16T11:43:42.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864460+0000", 2024-09-16T11:43:42.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6349999999999999e-06 2024-09-16T11:43:42.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879636+0000", 2024-09-16T11:43:42.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015175525 2024-09-16T11:43:42.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879677+0000", 2024-09-16T11:43:42.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1338999999999998e-05 2024-09-16T11:43:42.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879723+0000", 2024-09-16T11:43:42.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6066999999999998e-05 2024-09-16T11:43:42.309 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.309 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.309 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2098 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 2392064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864464+0000", 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4228993409999999, 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015370959, 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2098 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.310 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864464+0000", 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864464+0000", 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864464+0000", 2024-09-16T11:43:42.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-16T11:43:42.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864466+0000", 2024-09-16T11:43:42.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.609e-06 2024-09-16T11:43:42.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864471+0000", 2024-09-16T11:43:42.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5940000000000001e-06 2024-09-16T11:43:42.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864475+0000", 2024-09-16T11:43:42.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3249999999999999e-06 2024-09-16T11:43:42.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879741+0000", 2024-09-16T11:43:42.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.01526657 2024-09-16T11:43:42.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879786+0000", 2024-09-16T11:43:42.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4552000000000003e-05 2024-09-16T11:43:42.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879835+0000", 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2099 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 2342912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864478+0000", 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4228847509999998, 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015474740000000001, 2024-09-16T11:43:42.314 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2099 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864478+0000", 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864478+0000", 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864479+0000", 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864480+0000", 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.48e-06 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864485+0000", 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-06 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864488+0000", 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.084e-06 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879859+0000", 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015370582000000001 2024-09-16T11:43:42.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879905+0000", 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5819999999999998e-05 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879953+0000", 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8353999999999998e-05 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.319 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2100 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 2834432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.319 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864491+0000", 2024-09-16T11:43:42.319 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4228715489999999, 2024-09-16T11:43:42.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015571742, 2024-09-16T11:43:42.319 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.319 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.319 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.319 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.319 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.319 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2100 2024-09-16T11:43:42.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.319 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864491+0000", 2024-09-16T11:43:42.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864491+0000", 2024-09-16T11:43:42.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864492+0000", 2024-09-16T11:43:42.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864493+0000", 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4559999999999999e-06 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864498+0000", 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864501+0000", 2024-09-16T11:43:42.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-16T11:43:42.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879977+0000", 2024-09-16T11:43:42.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015475922 2024-09-16T11:43:42.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880018+0000", 2024-09-16T11:43:42.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1097999999999999e-05 2024-09-16T11:43:42.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880063+0000", 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4793000000000003e-05 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2101 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 2539520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864505+0000", 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4228580009999998, 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015665767000000001, 2024-09-16T11:43:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2101 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864505+0000", 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864505+0000", 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864505+0000", 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3099999999999999e-07 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864518+0000", 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3098000000000001e-05 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864524+0000", 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8409999999999996e-06 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864532+0000", 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.435e-06 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880086+0000", 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015553882999999999 2024-09-16T11:43:42.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880122+0000", 2024-09-16T11:43:42.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6884000000000001e-05 2024-09-16T11:43:42.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880171+0000", 2024-09-16T11:43:42.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294999999999998e-05 2024-09-16T11:43:42.327 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.327 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.327 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2103 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 2736128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864536+0000", 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4228273709999999, 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015759769999999999, 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2103 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864536+0000", 2024-09-16T11:43:42.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864536+0000", 2024-09-16T11:43:42.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864536+0000", 2024-09-16T11:43:42.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0900000000000002e-07 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864538+0000", 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6810000000000001e-06 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864543+0000", 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6729999999999997e-06 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864547+0000", 2024-09-16T11:43:42.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3110000000000001e-06 2024-09-16T11:43:42.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880194+0000", 2024-09-16T11:43:42.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015647449000000001 2024-09-16T11:43:42.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880235+0000", 2024-09-16T11:43:42.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0547999999999999e-05 2024-09-16T11:43:42.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880295+0000", 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0699000000000003e-05 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2173 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 917504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.865914+0000", 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4214490930000001, 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015115216000000001, 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.333 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2173 2024-09-16T11:43:42.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.333 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865914+0000", 2024-09-16T11:43:42.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865914+0000", 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865914+0000", 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865915+0000", 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865918+0000", 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8260000000000001e-06 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865925+0000", 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1230000000000002e-06 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880858+0000", 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014932822 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880938+0000", 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9979000000000001e-05 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881029+0000", 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1340000000000003e-05 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.336 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2174 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.337 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.865928+0000", 2024-09-16T11:43:42.337 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4214354139999998, 2024-09-16T11:43:42.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015289802, 2024-09-16T11:43:42.337 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.337 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.337 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2174 2024-09-16T11:43:42.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.337 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865928+0000", 2024-09-16T11:43:42.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865928+0000", 2024-09-16T11:43:42.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865928+0000", 2024-09-16T11:43:42.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-09-16T11:43:42.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865930+0000", 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.623e-06 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865935+0000", 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.507e-06 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865938+0000", 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.672e-06 2024-09-16T11:43:42.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881057+0000", 2024-09-16T11:43:42.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015119099 2024-09-16T11:43:42.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881132+0000", 2024-09-16T11:43:42.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4915999999999999e-05 2024-09-16T11:43:42.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881217+0000", 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5646e-05 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2175 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 1310720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.865950+0000", 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.421412745, 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015476405, 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2175 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865950+0000", 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865950+0000", 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865951+0000", 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2699999999999999e-07 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865953+0000", 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.762e-06 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865958+0000", 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6200000000000004e-06 2024-09-16T11:43:42.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865961+0000", 2024-09-16T11:43:42.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1e-06 2024-09-16T11:43:42.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881255+0000", 2024-09-16T11:43:42.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015293263 2024-09-16T11:43:42.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881331+0000", 2024-09-16T11:43:42.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6922000000000007e-05 2024-09-16T11:43:42.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881427+0000", 2024-09-16T11:43:42.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5210999999999994e-05 2024-09-16T11:43:42.345 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.345 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.345 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2176 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.865965+0000", 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4213977870000001, 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.01565888, 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2176 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.346 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865965+0000", 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865965+0000", 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865966+0000", 2024-09-16T11:43:42.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865967+0000", 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.59e-06 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865972+0000", 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2490000000000001e-06 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865977+0000", 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.262e-06 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881469+0000", 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015491868000000001 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881558+0000", 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9963000000000006e-05 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881624+0000", 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5616000000000003e-05 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2177 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.865981+0000", 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.421382506, 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015780228, 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.350 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2177 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865981+0000", 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865981+0000", 2024-09-16T11:43:42.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865981+0000", 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865982+0000", 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5239999999999999e-06 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865995+0000", 2024-09-16T11:43:42.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2797e-05 2024-09-16T11:43:42.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866006+0000", 2024-09-16T11:43:42.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0929e-05 2024-09-16T11:43:42.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881664+0000", 2024-09-16T11:43:42.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015657602999999999 2024-09-16T11:43:42.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881708+0000", 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4367999999999999e-05 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881761+0000", 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2679999999999997e-05 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2178 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.354 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866011+0000", 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.421352089, 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015888085999999999, 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2178 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866011+0000", 2024-09-16T11:43:42.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866011+0000", 2024-09-16T11:43:42.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866011+0000", 2024-09-16T11:43:42.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2e-07 2024-09-16T11:43:42.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866013+0000", 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9439999999999999e-06 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866020+0000", 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.861e-06 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866023+0000", 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1669999999999999e-06 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881795+0000", 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015771912999999999 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881849+0000", 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.401e-05 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881899+0000", 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9771000000000002e-05 2024-09-16T11:43:42.358 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.359 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.359 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2179 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 2146304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.359 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866034+0000", 2024-09-16T11:43:42.359 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4213287889999999, 2024-09-16T11:43:42.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.016016827000000001, 2024-09-16T11:43:42.359 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.359 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2179 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866034+0000", 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866034+0000", 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866035+0000", 2024-09-16T11:43:42.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:43:42.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866037+0000", 2024-09-16T11:43:42.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8920000000000001e-06 2024-09-16T11:43:42.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866043+0000", 2024-09-16T11:43:42.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0079999999999999e-06 2024-09-16T11:43:42.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866056+0000", 2024-09-16T11:43:42.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3841e-05 2024-09-16T11:43:42.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881932+0000", 2024-09-16T11:43:42.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.015875490999999999 2024-09-16T11:43:42.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.881992+0000", 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0418999999999999e-05 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.882051+0000", 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8755000000000002e-05 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2180 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 2588672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.363 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866061+0000", 2024-09-16T11:43:42.364 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.42130197, 2024-09-16T11:43:42.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.016124580999999999, 2024-09-16T11:43:42.364 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.364 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.364 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2180 2024-09-16T11:43:42.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.364 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866061+0000", 2024-09-16T11:43:42.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866061+0000", 2024-09-16T11:43:42.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866061+0000", 2024-09-16T11:43:42.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:43:42.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866068+0000", 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.827e-06 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866075+0000", 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6579999999999996e-06 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866079+0000", 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5829999999999998e-06 2024-09-16T11:43:42.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.882079+0000", 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.016000889000000001 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.882122+0000", 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2905999999999999e-05 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.882186+0000", 2024-09-16T11:43:42.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3281000000000006e-05 2024-09-16T11:43:42.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.368 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.368 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2181 4.1b 4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5 [read 2785280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.368 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866083+0000", 2024-09-16T11:43:42.368 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4212805099999999, 2024-09-16T11:43:42.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.016323522, 2024-09-16T11:43:42.368 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.368 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2181 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866083+0000", 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866083+0000", 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866083+0000", 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0900000000000002e-07 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866085+0000", 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866090+0000", 2024-09-16T11:43:42.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3940000000000004e-06 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866093+0000", 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4259999999999999e-06 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.882218+0000", 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.016125033 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.882310+0000", 2024-09-16T11:43:42.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1874999999999997e-05 2024-09-16T11:43:42.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.882406+0000", 2024-09-16T11:43:42.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5785000000000005e-05 2024-09-16T11:43:42.372 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.372 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.372 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.372 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:5182 4.13 4:cb2d65ef:::rbd_data.11c18a2497a7.000000000000000e:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:42.372 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:41.892500+0000", 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.394863032, 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.017072331999999999, 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5182 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.892500+0000", 2024-09-16T11:43:42.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.892500+0000", 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.892501+0000", 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4e-07 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.892507+0000", 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0329999999999997e-06 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.892516+0000", 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9959999999999999e-06 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.892520+0000", 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.892554+0000", 2024-09-16T11:43:42.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4453999999999999e-05 2024-09-16T11:43:42.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.892623+0000", 2024-09-16T11:43:42.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9366e-05 2024-09-16T11:43:42.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:42.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.895472+0000", 2024-09-16T11:43:42.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028485279999999999 2024-09-16T11:43:42.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.909485+0000", 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014013408 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.909496+0000", 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0613e-05 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.909531+0000", 2024-09-16T11:43:42.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5203000000000002e-05 2024-09-16T11:43:42.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:41.909572+0000", 2024-09-16T11:43:42.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1111e-05 2024-09-16T11:43:42.378 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.378 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.378 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.378 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:5404 4.9 4:965af514:::rbd_data.11c18a2497a7.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:42.378 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.058657+0000", 2024-09-16T11:43:42.378 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.228706408, 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023045708000000002, 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5404 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.058657+0000", 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.058657+0000", 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.058657+0000", 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1099999999999995e-07 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.058661+0000", 2024-09-16T11:43:42.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7730000000000001e-06 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.058670+0000", 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0189999999999995e-06 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.058674+0000", 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8709999999999999e-06 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.058702+0000", 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8147e-05 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.058747+0000", 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5042e-05 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.062489+0000", 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037415679999999998 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.081605+0000", 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019116129999999999 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.081637+0000", 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2447000000000003e-05 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.081674+0000", 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6650999999999999e-05 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.081702+0000", 2024-09-16T11:43:42.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8348999999999999e-05 2024-09-16T11:43:42.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.384 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.384 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.384 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:42.386 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:43:42.387 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:42.384+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:42.387 INFO:teuthology.orchestra.run.smithi070.stdout:728 write 0xe5d77f8 thru 0xe5e6b34 (0xf33d bytes) 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:733 write 0x3e5721 thru 0x3ec649 (0x6f29 bytes) 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:735 trunc from 0xe5e6b35 to 0x7af839c 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:738 flatten 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:740 read 0x15036af thru 0x150e4d9 (0xae2b bytes) 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:741 read 0x330d4a8 thru 0x3310a37 (0x3590 bytes) 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:742 writesame 0xd3cce45 thru 0xd3d079f data_size 0x1(0x395b bytes) 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:743 write 0xab42b57 thru 0xab44651 (0x1afb bytes) 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:744 trunc from 0xd3d07a0 to 0x311ad76 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:745 read 0x958d3 thru 0x98d0f (0x343d bytes) 2024-09-16T11:43:42.388 INFO:teuthology.orchestra.run.smithi070.stdout:746 read 0x1da368e thru 0x1dab1e2 (0x7b55 bytes) 2024-09-16T11:43:42.393 INFO:teuthology.orchestra.run.smithi070.stdout:748 write 0xe9472ff thru 0xe95155c (0xa25e bytes) 2024-09-16T11:43:42.396 INFO:teuthology.orchestra.run.smithi070.stdout:749 write 0x4b4086b thru 0x4b42564 (0x1cfa bytes) 2024-09-16T11:43:42.399 INFO:teuthology.orchestra.run.smithi070.stdout:750 writesame 0x1cb692d thru 0x1cb9de2 data_size 0x27(0x34b6 bytes) 2024-09-16T11:43:42.400 INFO:teuthology.orchestra.run.smithi070.stdout:752 writesame 0x2ff909 thru 0x30bd93 data_size 0x1d(0xc48b bytes) 2024-09-16T11:43:42.402 INFO:teuthology.orchestra.run.smithi070.stdout:753 read 0x7fe3a72 thru 0x7fee22a (0xa7b9 bytes) 2024-09-16T11:43:42.402 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.407 INFO:teuthology.orchestra.run.smithi070.stdout:755 trunc from 0xe95155d to 0x9d6ae2c 2024-09-16T11:43:42.416 INFO:teuthology.orchestra.run.smithi070.stdout:756 writesame 0x95f99aa thru 0x95fe14c data_size 0x1(0x47a3 bytes) 2024-09-16T11:43:42.425 INFO:teuthology.orchestra.run.smithi070.stdout:757 writesame 0x99ee8f3 thru 0x99f5692 data_size 0x10(0x6da0 bytes) 2024-09-16T11:43:42.429 INFO:teuthology.orchestra.run.smithi070.stdout:758 read 0x458a2b6 thru 0x4598315 (0xe060 bytes) 2024-09-16T11:43:42.429 INFO:teuthology.orchestra.run.smithi070.stdout:759 read 0x66fe393 thru 0x670251c (0x418a bytes) 2024-09-16T11:43:42.437 INFO:teuthology.orchestra.run.smithi070.stdout:760 writesame 0xc0e2bb8 thru 0xc0f002b data_size 0x2(0xd474 bytes) 2024-09-16T11:43:42.438 INFO:teuthology.orchestra.run.smithi070.stdout:761 punch from 0xa74a061 to 0xa74a1fb, (0x19a bytes) 2024-09-16T11:43:42.438 INFO:teuthology.orchestra.run.smithi070.stdout:762 read 0x87d792a thru 0x87df9be (0x8095 bytes) 2024-09-16T11:43:42.438 INFO:teuthology.orchestra.run.smithi070.stdout:763 write 0x97736a6 thru 0x97761e5 (0x2b40 bytes) 2024-09-16T11:43:42.440 INFO:teuthology.orchestra.run.smithi070.stdout:764 punch from 0x15b5ef6 to 0x15bba42, (0x5b4c bytes) 2024-09-16T11:43:42.440 INFO:teuthology.orchestra.run.smithi070.stdout:766 writesame 0x3002a3e thru 0x3010078 data_size 0x65(0xd63b bytes) 2024-09-16T11:43:42.441 INFO:teuthology.orchestra.run.smithi070.stdout:767 read 0x4dfdc8f thru 0x4e0d8b2 (0xfc24 bytes) 2024-09-16T11:43:42.441 INFO:teuthology.orchestra.run.smithi070.stdout:768 punch from 0xb7175d0 to 0xb723e79, (0xc8a9 bytes) 2024-09-16T11:43:42.442 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.442 INFO:teuthology.orchestra.run.smithi070.stdout:770 read 0x9c777b5 thru 0x9c80a90 (0x92dc bytes) 2024-09-16T11:43:42.450 INFO:teuthology.orchestra.run.smithi070.stdout:771 trunc from 0xc0f002c to 0x42c2f79 2024-09-16T11:43:42.460 INFO:teuthology.orchestra.run.smithi070.stdout:772 punch from 0x3b0f7c2 to 0x3b0ff5c, (0x79a bytes) 2024-09-16T11:43:42.470 INFO:teuthology.orchestra.run.smithi070.stdout:773 trunc from 0x42c2f79 to 0xcda75d2 2024-09-16T11:43:42.476 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.483 INFO:teuthology.orchestra.run.smithi070.stdout:776 trunc from 0xcda75d2 to 0x6d9abba 2024-09-16T11:43:42.486 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:42.485+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:42.496 INFO:teuthology.orchestra.run.smithi070.stdout:777 write 0x8198fbb thru 0x819f5ee (0x6634 bytes) 2024-09-16T11:43:42.508 INFO:teuthology.orchestra.run.smithi070.stdout:778 write 0xb4b236c thru 0xb4bdff2 (0xbc87 bytes) 2024-09-16T11:43:42.512 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.512 INFO:teuthology.orchestra.run.smithi070.stdout:781 punch from 0x381ccb9 to 0x382a22e, (0xd575 bytes) 2024-09-16T11:43:42.512 INFO:teuthology.orchestra.run.smithi070.stdout:782 write 0x836a52c thru 0x837858e (0xe063 bytes) 2024-09-16T11:43:42.514 INFO:teuthology.orchestra.run.smithi070.stdout:783 punch from 0x859a22a to 0x85a2bfa, (0x89d0 bytes) 2024-09-16T11:43:42.514 INFO:teuthology.orchestra.run.smithi070.stdout:784 write 0x67967d7 thru 0x6797800 (0x102a bytes) 2024-09-16T11:43:42.517 INFO:teuthology.orchestra.run.smithi070.stdout:786 read 0x7728a36 thru 0x773104e (0x8619 bytes) 2024-09-16T11:43:42.523 INFO:teuthology.orchestra.run.smithi070.stdout:787 write 0xbcd95c2 thru 0xbce3644 (0xa083 bytes) 2024-09-16T11:43:42.526 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.526 INFO:teuthology.orchestra.run.smithi070.stdout:790 write 0x36b3829 thru 0x36bb809 (0x7fe1 bytes) 2024-09-16T11:43:42.528 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.528 INFO:teuthology.orchestra.run.smithi070.stdout:793 write 0x4b21c88 thru 0x4b268ea (0x4c63 bytes) 2024-09-16T11:43:42.530 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:42.530 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:43:42.530 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:42.537 INFO:teuthology.orchestra.run.smithi070.stdout:797 writesame 0xc9bde80 thru 0xc9bee58 data_size 0x1(0xfd9 bytes) 2024-09-16T11:43:42.538 INFO:teuthology.orchestra.run.smithi070.stdout:800 read 0xa219b8c thru 0xa224b8a (0xafff bytes) 2024-09-16T11:43:42.540 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:43:42.543 INFO:teuthology.orchestra.run.smithi070.stdout:801 write 0xd200e15 thru 0xd202314 (0x1500 bytes) 2024-09-16T11:43:42.552 INFO:teuthology.orchestra.run.smithi070.stdout:802 trunc from 0xd202315 to 0x876b067 2024-09-16T11:43:42.563 INFO:teuthology.orchestra.run.smithi070.stdout:806 read 0x2078c37 thru 0x207d7d7 (0x4ba1 bytes) 2024-09-16T11:43:42.569 INFO:teuthology.orchestra.run.smithi070.stdout:807 trunc from 0x876b067 to 0x4066113 2024-09-16T11:43:42.580 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.587 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:42.585+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:42.590 INFO:teuthology.orchestra.run.smithi070.stdout:810 write 0x88560cc thru 0x885a47c (0x43b1 bytes) 2024-09-16T11:43:42.593 INFO:teuthology.orchestra.run.smithi070.stdout:811 writesame 0x7db74e3 thru 0x7dc4888 data_size 0x1(0xd3a6 bytes) 2024-09-16T11:43:42.611 INFO:teuthology.orchestra.run.smithi070.stdout:812 punch from 0x49d04d9 to 0x49dbad9, (0xb600 bytes) 2024-09-16T11:43:42.611 INFO:teuthology.orchestra.run.smithi070.stdout:814 write 0x6264483 thru 0x6266182 (0x1d00 bytes) 2024-09-16T11:43:42.625 INFO:teuthology.orchestra.run.smithi070.stdout:815 write 0xc727647 thru 0xc730821 (0x91db bytes) 2024-09-16T11:43:42.638 INFO:teuthology.orchestra.run.smithi070.stdout:817 trunc from 0xc730822 to 0x4616e1a 2024-09-16T11:43:42.648 INFO:teuthology.orchestra.run.smithi070.stdout:818 writesame 0x3c68870 thru 0x3c6c8e3 data_size 0x6e(0x4074 bytes) 2024-09-16T11:43:42.651 INFO:teuthology.orchestra.run.smithi070.stdout:819 read 0xfa97dc thru 0xfb8259 (0xea7e bytes) 2024-09-16T11:43:42.652 INFO:teuthology.orchestra.run.smithi070.stdout:821 write 0x2bb0800 thru 0x2bb6c7b (0x647c bytes) 2024-09-16T11:43:42.669 INFO:teuthology.orchestra.run.smithi070.stdout:822 write 0xc48628d thru 0xc48a244 (0x3fb8 bytes) 2024-09-16T11:43:42.680 INFO:teuthology.orchestra.run.smithi070.stdout:823 writesame 0xecd48c0 thru 0xecd959a data_size 0x5(0x4cdb bytes) 2024-09-16T11:43:42.681 INFO:teuthology.orchestra.run.smithi070.stdout:825 read 0x4eb6246 thru 0x4ec41e9 (0xdfa4 bytes) 2024-09-16T11:43:42.681 INFO:teuthology.orchestra.run.smithi070.stdout:826 read 0x3410c7c thru 0x341cdc5 (0xc14a bytes) 2024-09-16T11:43:42.681 INFO:teuthology.orchestra.run.smithi070.stdout:828 punch from 0x4ecacae to 0x4ed6435, (0xb787 bytes) 2024-09-16T11:43:42.681 INFO:teuthology.orchestra.run.smithi070.stdout:829 write 0xe5500a9 thru 0xe5577e2 (0x773a bytes) 2024-09-16T11:43:42.684 INFO:teuthology.orchestra.run.smithi070.stdout:830 punch from 0x3ccdb8d to 0x3cd27c6, (0x4c39 bytes) 2024-09-16T11:43:42.684 INFO:teuthology.orchestra.run.smithi070.stdout:831 write 0xb72e867 thru 0xb737a5e (0x91f8 bytes) 2024-09-16T11:43:42.685 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:42.685 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:42.686 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:42.686 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:42.686 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:42.688 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:42.686+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:42.694 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:43:42.701 INFO:teuthology.orchestra.run.smithi070.stdout:832 trunc from 0xecd959b to 0x2d1c401 2024-09-16T11:43:42.727 INFO:teuthology.orchestra.run.smithi070.stdout:833 write 0xa38766a thru 0xa3880cd (0xa64 bytes) 2024-09-16T11:43:42.729 INFO:teuthology.orchestra.run.smithi070.stdout:836 read 0x642b49b thru 0x6434baa (0x9710 bytes) 2024-09-16T11:43:42.729 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.730 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.730 INFO:teuthology.orchestra.run.smithi070.stdout:840 punch from 0x9e4b384 to 0x9e5256b, (0x71e7 bytes) 2024-09-16T11:43:42.730 INFO:teuthology.orchestra.run.smithi070.stdout:843 write 0x29f17d9 thru 0x29fe0bd (0xc8e5 bytes) 2024-09-16T11:43:42.742 INFO:teuthology.orchestra.run.smithi070.stdout:844 write 0xd5679d2 thru 0xd572532 (0xab61 bytes) 2024-09-16T11:43:42.745 INFO:teuthology.orchestra.run.smithi070.stdout:845 write 0x81de576 thru 0x81def51 (0x9dc bytes) 2024-09-16T11:43:42.747 INFO:teuthology.orchestra.run.smithi070.stdout:847 writesame 0xb078f2b thru 0xb0800c5 data_size 0x1(0x719b bytes) 2024-09-16T11:43:42.762 INFO:teuthology.orchestra.run.smithi070.stdout:848 read 0x3c1e53e thru 0x3c2777a (0x923d bytes) 2024-09-16T11:43:42.777 INFO:teuthology.orchestra.run.smithi070.stdout:849 trunc from 0xd572533 to 0x590f1d 2024-09-16T11:43:42.790 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:42.788+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:42.801 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:42.801 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:43:42.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:43:42.802 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:43:42.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.802 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2119 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2375680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.802 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864670+0000", 2024-09-16T11:43:42.802 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9341929489999998, 2024-09-16T11:43:42.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013064175000000001, 2024-09-16T11:43:42.802 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.802 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.802 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2119 2024-09-16T11:43:42.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.803 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864670+0000", 2024-09-16T11:43:42.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864670+0000", 2024-09-16T11:43:42.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864670+0000", 2024-09-16T11:43:42.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:43:42.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864672+0000", 2024-09-16T11:43:42.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.691e-06 2024-09-16T11:43:42.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864677+0000", 2024-09-16T11:43:42.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6860000000000003e-06 2024-09-16T11:43:42.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864681+0000", 2024-09-16T11:43:42.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-06 2024-09-16T11:43:42.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877539+0000", 2024-09-16T11:43:42.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012858748999999999 2024-09-16T11:43:42.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877632+0000", 2024-09-16T11:43:42.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2367999999999994e-05 2024-09-16T11:43:42.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877734+0000", 2024-09-16T11:43:42.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102 2024-09-16T11:43:42.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.807 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.807 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2120 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.807 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864685+0000", 2024-09-16T11:43:42.807 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9341779610000001, 2024-09-16T11:43:42.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013171611999999999, 2024-09-16T11:43:42.807 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.807 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.808 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2120 2024-09-16T11:43:42.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.808 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864685+0000", 2024-09-16T11:43:42.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864685+0000", 2024-09-16T11:43:42.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864685+0000", 2024-09-16T11:43:42.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-16T11:43:42.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864692+0000", 2024-09-16T11:43:42.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5039999999999999e-06 2024-09-16T11:43:42.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864697+0000", 2024-09-16T11:43:42.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.699e-06 2024-09-16T11:43:42.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864701+0000", 2024-09-16T11:43:42.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6009999999999999e-06 2024-09-16T11:43:42.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877756+0000", 2024-09-16T11:43:42.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0130554 2024-09-16T11:43:42.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877801+0000", 2024-09-16T11:43:42.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5157000000000002e-05 2024-09-16T11:43:42.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877856+0000", 2024-09-16T11:43:42.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4910000000000001e-05 2024-09-16T11:43:42.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.812 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.812 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2121 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2818048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.812 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864708+0000", 2024-09-16T11:43:42.812 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.934154919, 2024-09-16T11:43:42.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013354938, 2024-09-16T11:43:42.813 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.813 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.813 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2121 2024-09-16T11:43:42.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.813 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864708+0000", 2024-09-16T11:43:42.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864708+0000", 2024-09-16T11:43:42.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864708+0000", 2024-09-16T11:43:42.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:43:42.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864710+0000", 2024-09-16T11:43:42.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:43:42.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864716+0000", 2024-09-16T11:43:42.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7289999999999997e-06 2024-09-16T11:43:42.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864720+0000", 2024-09-16T11:43:42.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9990000000000002e-06 2024-09-16T11:43:42.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877883+0000", 2024-09-16T11:43:42.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013162903 2024-09-16T11:43:42.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877969+0000", 2024-09-16T11:43:42.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6027999999999997e-05 2024-09-16T11:43:42.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878063+0000", 2024-09-16T11:43:42.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4086999999999996e-05 2024-09-16T11:43:42.817 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.817 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.817 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.817 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2122 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.817 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864723+0000", 2024-09-16T11:43:42.818 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.93413967, 2024-09-16T11:43:42.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013470271000000001, 2024-09-16T11:43:42.818 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.818 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.818 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.818 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.818 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.818 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2122 2024-09-16T11:43:42.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.818 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864723+0000", 2024-09-16T11:43:42.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864723+0000", 2024-09-16T11:43:42.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864723+0000", 2024-09-16T11:43:42.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6199999999999999e-07 2024-09-16T11:43:42.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864725+0000", 2024-09-16T11:43:42.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5960000000000001e-06 2024-09-16T11:43:42.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864730+0000", 2024-09-16T11:43:42.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1950000000000002e-06 2024-09-16T11:43:42.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864733+0000", 2024-09-16T11:43:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-06 2024-09-16T11:43:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878093+0000", 2024-09-16T11:43:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013359473 2024-09-16T11:43:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878140+0000", 2024-09-16T11:43:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6692e-05 2024-09-16T11:43:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878193+0000", 2024-09-16T11:43:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3683000000000001e-05 2024-09-16T11:43:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.823 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2124 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2326528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.823 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864738+0000", 2024-09-16T11:43:42.823 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9341248760000003, 2024-09-16T11:43:42.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013679180000000001, 2024-09-16T11:43:42.823 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.823 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.823 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2124 2024-09-16T11:43:42.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.824 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864738+0000", 2024-09-16T11:43:42.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864738+0000", 2024-09-16T11:43:42.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864738+0000", 2024-09-16T11:43:42.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:43:42.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864740+0000", 2024-09-16T11:43:42.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5990000000000001e-06 2024-09-16T11:43:42.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864745+0000", 2024-09-16T11:43:42.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5600000000000001e-06 2024-09-16T11:43:42.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864749+0000", 2024-09-16T11:43:42.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2590000000000001e-06 2024-09-16T11:43:42.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878230+0000", 2024-09-16T11:43:42.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013481215 2024-09-16T11:43:42.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878314+0000", 2024-09-16T11:43:42.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4037999999999997e-05 2024-09-16T11:43:42.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878417+0000", 2024-09-16T11:43:42.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010315900000000001 2024-09-16T11:43:42.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.827 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.828 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.828 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2125 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2523136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.828 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864752+0000", 2024-09-16T11:43:42.828 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9341107719999999, 2024-09-16T11:43:42.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013813690999999999, 2024-09-16T11:43:42.828 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.828 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.829 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2125 2024-09-16T11:43:42.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.829 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864752+0000", 2024-09-16T11:43:42.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864752+0000", 2024-09-16T11:43:42.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864752+0000", 2024-09-16T11:43:42.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:43:42.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864754+0000", 2024-09-16T11:43:42.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:43:42.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864760+0000", 2024-09-16T11:43:42.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5119999999999997e-06 2024-09-16T11:43:42.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864764+0000", 2024-09-16T11:43:42.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5779999999999999e-06 2024-09-16T11:43:42.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878448+0000", 2024-09-16T11:43:42.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013683749 2024-09-16T11:43:42.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878496+0000", 2024-09-16T11:43:42.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8028999999999997e-05 2024-09-16T11:43:42.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878566+0000", 2024-09-16T11:43:42.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9900000000000005e-05 2024-09-16T11:43:42.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.833 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.833 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2126 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2719744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.833 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864767+0000", 2024-09-16T11:43:42.833 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9340955969999998, 2024-09-16T11:43:42.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013991063999999999, 2024-09-16T11:43:42.833 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.833 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.833 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.834 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2126 2024-09-16T11:43:42.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.834 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864767+0000", 2024-09-16T11:43:42.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864767+0000", 2024-09-16T11:43:42.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864767+0000", 2024-09-16T11:43:42.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1800000000000002e-07 2024-09-16T11:43:42.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864769+0000", 2024-09-16T11:43:42.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.53e-06 2024-09-16T11:43:42.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864772+0000", 2024-09-16T11:43:42.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4580000000000002e-06 2024-09-16T11:43:42.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864774+0000", 2024-09-16T11:43:42.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.979e-06 2024-09-16T11:43:42.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878598+0000", 2024-09-16T11:43:42.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013823821999999999 2024-09-16T11:43:42.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878673+0000", 2024-09-16T11:43:42.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4476999999999999e-05 2024-09-16T11:43:42.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878758+0000", 2024-09-16T11:43:42.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-05 2024-09-16T11:43:42.838 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.838 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.838 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.838 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2182 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 65536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.838 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.865997+0000", 2024-09-16T11:43:42.838 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9328658130000003, 2024-09-16T11:43:42.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012930596000000001, 2024-09-16T11:43:42.839 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.839 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.839 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.839 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.839 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.839 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2182 2024-09-16T11:43:42.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.839 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865997+0000", 2024-09-16T11:43:42.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865997+0000", 2024-09-16T11:43:42.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865998+0000", 2024-09-16T11:43:42.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:43:42.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866001+0000", 2024-09-16T11:43:42.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7220000000000001e-06 2024-09-16T11:43:42.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866013+0000", 2024-09-16T11:43:42.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.188e-05 2024-09-16T11:43:42.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866026+0000", 2024-09-16T11:43:42.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2938999999999999e-05 2024-09-16T11:43:42.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878795+0000", 2024-09-16T11:43:42.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012769024 2024-09-16T11:43:42.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878864+0000", 2024-09-16T11:43:42.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8427000000000001e-05 2024-09-16T11:43:42.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878927+0000", 2024-09-16T11:43:42.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3746999999999998e-05 2024-09-16T11:43:42.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.843 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.844 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2183 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 262144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.844 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866033+0000", 2024-09-16T11:43:42.844 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9328291960000001, 2024-09-16T11:43:42.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013099107, 2024-09-16T11:43:42.844 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.844 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.844 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2183 2024-09-16T11:43:42.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.845 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866033+0000", 2024-09-16T11:43:42.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866033+0000", 2024-09-16T11:43:42.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866034+0000", 2024-09-16T11:43:42.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:43:42.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866036+0000", 2024-09-16T11:43:42.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:43:42.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866047+0000", 2024-09-16T11:43:42.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1083999999999999e-05 2024-09-16T11:43:42.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866062+0000", 2024-09-16T11:43:42.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5156e-05 2024-09-16T11:43:42.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878961+0000", 2024-09-16T11:43:42.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012898687000000001 2024-09-16T11:43:42.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879036+0000", 2024-09-16T11:43:42.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4540000000000001e-05 2024-09-16T11:43:42.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879133+0000", 2024-09-16T11:43:42.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6887999999999998e-05 2024-09-16T11:43:42.848 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.848 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.849 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.849 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2184 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.849 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866067+0000", 2024-09-16T11:43:42.849 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.932795187, 2024-09-16T11:43:42.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013274744, 2024-09-16T11:43:42.849 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.849 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.849 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.849 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.849 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.850 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2184 2024-09-16T11:43:42.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.850 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866067+0000", 2024-09-16T11:43:42.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866067+0000", 2024-09-16T11:43:42.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866068+0000", 2024-09-16T11:43:42.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:43:42.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866070+0000", 2024-09-16T11:43:42.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-16T11:43:42.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866076+0000", 2024-09-16T11:43:42.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1809999999999998e-06 2024-09-16T11:43:42.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866080+0000", 2024-09-16T11:43:42.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8279999999999996e-06 2024-09-16T11:43:42.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879170+0000", 2024-09-16T11:43:42.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013089747000000001 2024-09-16T11:43:42.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879252+0000", 2024-09-16T11:43:42.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2279000000000003e-05 2024-09-16T11:43:42.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879342+0000", 2024-09-16T11:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0195e-05 2024-09-16T11:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2185 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866084+0000", 2024-09-16T11:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9327788209999999, 2024-09-16T11:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013513447, 2024-09-16T11:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2185 2024-09-16T11:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866084+0000", 2024-09-16T11:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866084+0000", 2024-09-16T11:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866084+0000", 2024-09-16T11:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-09-16T11:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866086+0000", 2024-09-16T11:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6789999999999999e-06 2024-09-16T11:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866091+0000", 2024-09-16T11:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.626e-06 2024-09-16T11:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866095+0000", 2024-09-16T11:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879406+0000", 2024-09-16T11:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013311523 2024-09-16T11:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879497+0000", 2024-09-16T11:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0507999999999995e-05 2024-09-16T11:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879597+0000", 2024-09-16T11:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010027900000000001 2024-09-16T11:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2186 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866100+0000", 2024-09-16T11:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9327627860000001, 2024-09-16T11:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013635298000000001, 2024-09-16T11:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2186 2024-09-16T11:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866100+0000", 2024-09-16T11:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866100+0000", 2024-09-16T11:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866100+0000", 2024-09-16T11:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.15e-07 2024-09-16T11:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866109+0000", 2024-09-16T11:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8189999999999998e-06 2024-09-16T11:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866115+0000", 2024-09-16T11:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2770000000000002e-06 2024-09-16T11:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866121+0000", 2024-09-16T11:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1279999999999996e-06 2024-09-16T11:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879618+0000", 2024-09-16T11:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013496869 2024-09-16T11:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879676+0000", 2024-09-16T11:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8075000000000002e-05 2024-09-16T11:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879735+0000", 2024-09-16T11:43:42.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8715000000000003e-05 2024-09-16T11:43:42.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.867 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.867 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2187 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.867 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866125+0000", 2024-09-16T11:43:42.867 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9327371219999998, 2024-09-16T11:43:42.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013740278999999999, 2024-09-16T11:43:42.868 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.868 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.868 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2187 2024-09-16T11:43:42.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.868 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866125+0000", 2024-09-16T11:43:42.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866125+0000", 2024-09-16T11:43:42.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866126+0000", 2024-09-16T11:43:42.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.08e-07 2024-09-16T11:43:42.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866128+0000", 2024-09-16T11:43:42.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6160000000000001e-06 2024-09-16T11:43:42.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866145+0000", 2024-09-16T11:43:42.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.747e-05 2024-09-16T11:43:42.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866161+0000", 2024-09-16T11:43:42.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6382e-05 2024-09-16T11:43:42.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879767+0000", 2024-09-16T11:43:42.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013605894 2024-09-16T11:43:42.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879813+0000", 2024-09-16T11:43:42.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999999e-05 2024-09-16T11:43:42.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879866+0000", 2024-09-16T11:43:42.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3109000000000003e-05 2024-09-16T11:43:42.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.872 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.873 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2188 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1540096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.873 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866166+0000", 2024-09-16T11:43:42.873 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9326967870000002, 2024-09-16T11:43:42.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013814610999999999, 2024-09-16T11:43:42.873 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.873 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.874 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2188 2024-09-16T11:43:42.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.874 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866166+0000", 2024-09-16T11:43:42.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866166+0000", 2024-09-16T11:43:42.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866166+0000", 2024-09-16T11:43:42.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:43:42.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866181+0000", 2024-09-16T11:43:42.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.489e-05 2024-09-16T11:43:42.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866185+0000", 2024-09-16T11:43:42.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.386e-06 2024-09-16T11:43:42.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866188+0000", 2024-09-16T11:43:42.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.469e-06 2024-09-16T11:43:42.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879888+0000", 2024-09-16T11:43:42.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013700273000000001 2024-09-16T11:43:42.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879932+0000", 2024-09-16T11:43:42.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4039000000000003e-05 2024-09-16T11:43:42.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879980+0000", 2024-09-16T11:43:42.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8186000000000002e-05 2024-09-16T11:43:42.878 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.878 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.878 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.879 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2189 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.879 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866191+0000", 2024-09-16T11:43:42.879 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9326721070000001, 2024-09-16T11:43:42.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013900292999999999, 2024-09-16T11:43:42.879 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.880 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.880 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.880 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.880 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.880 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2189 2024-09-16T11:43:42.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.880 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866191+0000", 2024-09-16T11:43:42.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866191+0000", 2024-09-16T11:43:42.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866191+0000", 2024-09-16T11:43:42.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1699999999999999e-07 2024-09-16T11:43:42.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866193+0000", 2024-09-16T11:43:42.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3410000000000001e-06 2024-09-16T11:43:42.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866196+0000", 2024-09-16T11:43:42.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9840000000000001e-06 2024-09-16T11:43:42.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866198+0000", 2024-09-16T11:43:42.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8369999999999999e-06 2024-09-16T11:43:42.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880001+0000", 2024-09-16T11:43:42.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013803426000000001 2024-09-16T11:43:42.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880042+0000", 2024-09-16T11:43:42.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0410000000000001e-05 2024-09-16T11:43:42.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880091+0000", 2024-09-16T11:43:42.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8977999999999998e-05 2024-09-16T11:43:42.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.884 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.884 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2190 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1933312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.884 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866200+0000", 2024-09-16T11:43:42.884 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9326628289999999, 2024-09-16T11:43:42.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013997562, 2024-09-16T11:43:42.885 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.885 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.885 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2190 2024-09-16T11:43:42.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.885 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866200+0000", 2024-09-16T11:43:42.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866200+0000", 2024-09-16T11:43:42.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866200+0000", 2024-09-16T11:43:42.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1899999999999999e-07 2024-09-16T11:43:42.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866201+0000", 2024-09-16T11:43:42.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0249999999999999e-06 2024-09-16T11:43:42.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866204+0000", 2024-09-16T11:43:42.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.858e-06 2024-09-16T11:43:42.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866206+0000", 2024-09-16T11:43:42.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:43:42.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880111+0000", 2024-09-16T11:43:42.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013905463999999999 2024-09-16T11:43:42.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880151+0000", 2024-09-16T11:43:42.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0182999999999998e-05 2024-09-16T11:43:42.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880197+0000", 2024-09-16T11:43:42.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6050000000000001e-05 2024-09-16T11:43:42.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.889 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.890 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2191 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2129920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.890 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866207+0000", 2024-09-16T11:43:42.890 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9326551749999998, 2024-09-16T11:43:42.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014099525, 2024-09-16T11:43:42.890 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.890 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.890 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2191 2024-09-16T11:43:42.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.890 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866207+0000", 2024-09-16T11:43:42.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866207+0000", 2024-09-16T11:43:42.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866208+0000", 2024-09-16T11:43:42.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-16T11:43:42.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866209+0000", 2024-09-16T11:43:42.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.519e-06 2024-09-16T11:43:42.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866212+0000", 2024-09-16T11:43:42.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8210000000000002e-06 2024-09-16T11:43:42.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866218+0000", 2024-09-16T11:43:42.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5280000000000003e-06 2024-09-16T11:43:42.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880218+0000", 2024-09-16T11:43:42.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014000383999999999 2024-09-16T11:43:42.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880259+0000", 2024-09-16T11:43:42.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0815000000000001e-05 2024-09-16T11:43:42.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880307+0000", 2024-09-16T11:43:42.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8241000000000001e-05 2024-09-16T11:43:42.895 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.895 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.895 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.895 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2192 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2572288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.895 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866249+0000", 2024-09-16T11:43:42.895 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.932614074, 2024-09-16T11:43:42.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014184179999999999, 2024-09-16T11:43:42.896 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.896 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.896 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.896 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.896 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.896 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2192 2024-09-16T11:43:42.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.896 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866249+0000", 2024-09-16T11:43:42.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866249+0000", 2024-09-16T11:43:42.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866249+0000", 2024-09-16T11:43:42.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:43:42.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866251+0000", 2024-09-16T11:43:42.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1119999999999999e-06 2024-09-16T11:43:42.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866257+0000", 2024-09-16T11:43:42.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9159999999999998e-06 2024-09-16T11:43:42.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866262+0000", 2024-09-16T11:43:42.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8910000000000003e-06 2024-09-16T11:43:42.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880330+0000", 2024-09-16T11:43:42.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014068429 2024-09-16T11:43:42.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880383+0000", 2024-09-16T11:43:42.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029000000000003e-05 2024-09-16T11:43:42.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880433+0000", 2024-09-16T11:43:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9446000000000001e-05 2024-09-16T11:43:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2193 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2768896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.901 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866266+0000", 2024-09-16T11:43:42.902 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9325962269999999, 2024-09-16T11:43:42.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.01427954, 2024-09-16T11:43:42.902 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.902 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:42.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.902 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2193 2024-09-16T11:43:42.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.902 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866266+0000", 2024-09-16T11:43:42.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866266+0000", 2024-09-16T11:43:42.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866267+0000", 2024-09-16T11:43:42.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7300000000000001e-07 2024-09-16T11:43:42.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866269+0000", 2024-09-16T11:43:42.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.73e-06 2024-09-16T11:43:42.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866275+0000", 2024-09-16T11:43:42.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0229999999999998e-06 2024-09-16T11:43:42.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866278+0000", 2024-09-16T11:43:42.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-16T11:43:42.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880454+0000", 2024-09-16T11:43:42.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014175755999999999 2024-09-16T11:43:42.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880498+0000", 2024-09-16T11:43:42.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4033999999999999e-05 2024-09-16T11:43:42.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880546+0000", 2024-09-16T11:43:42.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7849999999999998e-05 2024-09-16T11:43:42.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.907 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.907 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2670 4.15 4:aecbca69:::rbd_data.11c197cf718c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:42.908 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.995522+0000", 2024-09-16T11:43:42.908 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.8033401499999999, 2024-09-16T11:43:42.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024883367999999999, 2024-09-16T11:43:42.908 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:42.908 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:42.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:42.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:42.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:42.908 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2670 2024-09-16T11:43:42.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.908 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:42.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:42.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.995522+0000", 2024-09-16T11:43:42.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:42.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.995522+0000", 2024-09-16T11:43:42.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:42.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:42.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.995524+0000", 2024-09-16T11:43:42.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2810000000000001e-06 2024-09-16T11:43:42.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:42.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996603+0000", 2024-09-16T11:43:42.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010795329999999999 2024-09-16T11:43:42.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:42.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996616+0000", 2024-09-16T11:43:42.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2850999999999999e-05 2024-09-16T11:43:42.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:42.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996626+0000", 2024-09-16T11:43:42.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0362e-05 2024-09-16T11:43:42.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:42.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996664+0000", 2024-09-16T11:43:42.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7078e-05 2024-09-16T11:43:42.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:42.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996757+0000", 2024-09-16T11:43:42.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3614000000000002e-05 2024-09-16T11:43:42.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:42.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.001077+0000", 2024-09-16T11:43:42.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0043196570000000002 2024-09-16T11:43:42.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:42.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.020321+0000", 2024-09-16T11:43:42.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019243871999999999 2024-09-16T11:43:42.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:42.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.020330+0000", 2024-09-16T11:43:42.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2369999999999998e-06 2024-09-16T11:43:42.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:42.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.020369+0000", 2024-09-16T11:43:42.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9233999999999997e-05 2024-09-16T11:43:42.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:42.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:42.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:42.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.020406+0000", 2024-09-16T11:43:42.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6649000000000001e-05 2024-09-16T11:43:42.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.914 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:42.915 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:42.915 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:42.916 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T11:43:42.917 INFO:teuthology.orchestra.run.smithi070.stdout:850 write 0x8195173 thru 0x81a219a (0xd028 bytes) 2024-09-16T11:43:42.917 INFO:teuthology.orchestra.run.smithi070.stdout:851 punch from 0x75917e3 to 0x759f45e, (0xdc7b bytes) 2024-09-16T11:43:42.917 INFO:teuthology.orchestra.run.smithi070.stdout:852 punch from 0x4603e36 to 0x46044eb, (0x6b5 bytes) 2024-09-16T11:43:42.917 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.917 INFO:teuthology.orchestra.run.smithi070.stdout:854 trunc from 0x81a219b to 0xdcb00fc 2024-09-16T11:43:42.918 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:43:42.922 INFO:teuthology.orchestra.run.smithi070.stdout:856 write 0x1efc655 thru 0x1f0a904 (0xe2b0 bytes) 2024-09-16T11:43:42.922 INFO:teuthology.orchestra.run.smithi070.stdout:857 punch from 0xb613acb to 0xb621670, (0xdba5 bytes) 2024-09-16T11:43:42.922 INFO:teuthology.orchestra.run.smithi070.stdout:859 read 0x6b8f87 thru 0x6c615a (0xd1d4 bytes) 2024-09-16T11:43:42.922 INFO:teuthology.orchestra.run.smithi070.stdout:860 write 0x28c441f thru 0x28cdf4c (0x9b2e bytes) 2024-09-16T11:43:42.922 INFO:teuthology.orchestra.run.smithi070.stdout:861 write 0x9703528 thru 0x9707c87 (0x4760 bytes) 2024-09-16T11:43:42.922 INFO:teuthology.orchestra.run.smithi070.stdout:862 punch from 0x2e9a245 to 0x2ea10e4, (0x6e9f bytes) 2024-09-16T11:43:42.922 INFO:teuthology.orchestra.run.smithi070.stdout:863 write 0xc76099f thru 0xc766a71 (0x60d3 bytes) 2024-09-16T11:43:42.923 INFO:teuthology.orchestra.run.smithi070.stdout:866 write 0x74eceda thru 0x74fbd5e (0xee85 bytes) 2024-09-16T11:43:42.923 INFO:teuthology.orchestra.run.smithi070.stdout:867 write 0x45dbb3a thru 0x45e1259 (0x5720 bytes) 2024-09-16T11:43:42.923 INFO:teuthology.orchestra.run.smithi070.stdout:868 trunc from 0xdcb00fc to 0x5f266fe 2024-09-16T11:43:42.923 INFO:teuthology.orchestra.run.smithi070.stdout:869 read 0x2334c26 thru 0x233c01f (0x73fa bytes) 2024-09-16T11:43:42.923 INFO:teuthology.orchestra.run.smithi070.stdout:870 trunc from 0x5f266fe to 0xad377b4 2024-09-16T11:43:42.923 INFO:teuthology.orchestra.run.smithi070.stdout:872 read 0x49c1205 thru 0x49c8e32 (0x7c2e bytes) 2024-09-16T11:43:42.923 INFO:teuthology.orchestra.run.smithi070.stdout:873 read 0x45d951c thru 0x45e12ae (0x7d93 bytes) 2024-09-16T11:43:42.923 INFO:teuthology.orchestra.run.smithi070.stdout:874 write 0x1b06354 thru 0x1b09f21 (0x3bce bytes) 2024-09-16T11:43:42.923 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.923 INFO:teuthology.orchestra.run.smithi070.stdout:876 write 0x58248b thru 0x58827e (0x5df4 bytes) 2024-09-16T11:43:42.923 INFO:teuthology.orchestra.run.smithi070.stdout:877 write 0x9c95fd3 thru 0x9c96685 (0x6b3 bytes) 2024-09-16T11:43:42.924 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.924 INFO:teuthology.orchestra.run.smithi070.stdout:881 read 0x481afd2 thru 0x4822441 (0x7470 bytes) 2024-09-16T11:43:42.924 INFO:teuthology.orchestra.run.smithi070.stdout:882 punch from 0x926c639 to 0x92731c3, (0x6b8a bytes) 2024-09-16T11:43:42.924 INFO:teuthology.orchestra.run.smithi070.stdout:883 read 0x6d91a21 thru 0x6d99b62 (0x8142 bytes) 2024-09-16T11:43:42.924 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.924 INFO:teuthology.orchestra.run.smithi070.stdout:885 read 0x266b2f thru 0x268080 (0x1552 bytes) 2024-09-16T11:43:42.924 INFO:teuthology.orchestra.run.smithi070.stdout:886 read 0x5b5b039 thru 0x5b5be80 (0xe48 bytes) 2024-09-16T11:43:42.924 INFO:teuthology.orchestra.run.smithi070.stdout:887 write 0xd58079e thru 0xd586223 (0x5a86 bytes) 2024-09-16T11:43:42.924 INFO:teuthology.orchestra.run.smithi070.stdout:888 writesame 0xe8f0557 thru 0xe8ff1c4 data_size 0x35(0xec6e bytes) 2024-09-16T11:43:42.924 INFO:teuthology.orchestra.run.smithi070.stdout:889 write 0x510be3b thru 0x5117609 (0xb7cf bytes) 2024-09-16T11:43:42.924 INFO:teuthology.orchestra.run.smithi070.stdout:890 write 0x7817311 thru 0x7824913 (0xd603 bytes) 2024-09-16T11:43:42.925 INFO:teuthology.orchestra.run.smithi070.stdout:891 read 0xaab8e0b thru 0xaabefb9 (0x61af bytes) 2024-09-16T11:43:42.925 INFO:teuthology.orchestra.run.smithi070.stdout:893 punch from 0x546f9bc to 0x547c5c1, (0xcc05 bytes) 2024-09-16T11:43:42.925 INFO:teuthology.orchestra.run.smithi070.stdout:894 write 0xb503b8a thru 0xb50ea5c (0xaed3 bytes) 2024-09-16T11:43:42.925 INFO:teuthology.orchestra.run.smithi070.stdout:895 trunc from 0xe8ff1c5 to 0x7882dad 2024-09-16T11:43:42.925 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:42.914+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:42.926 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.926 INFO:teuthology.orchestra.run.smithi070.stdout:897 write 0x714e136 thru 0x7155ecd (0x7d98 bytes) 2024-09-16T11:43:42.926 INFO:teuthology.orchestra.run.smithi070.stdout:898 punch from 0x36e7786 to 0x36f0f1d, (0x9797 bytes) 2024-09-16T11:43:42.933 INFO:teuthology.orchestra.run.smithi070.stdout:900 writesame 0x87cca2f thru 0x87cde6b data_size 0xb(0x143d bytes) 2024-09-16T11:43:42.933 INFO:teuthology.orchestra.run.smithi070.stdout:901 write 0x48ac359 thru 0x48b1c83 (0x592b bytes) 2024-09-16T11:43:42.936 INFO:teuthology.orchestra.run.smithi070.stdout:903 read 0x3fc2a25 thru 0x3fcd90f (0xaeeb bytes) 2024-09-16T11:43:42.936 INFO:teuthology.orchestra.run.smithi070.stdout:904 write 0x16eaa9a thru 0x16f1986 (0x6eed bytes) 2024-09-16T11:43:42.946 INFO:teuthology.orchestra.run.smithi070.stdout:905 trunc from 0x87cde6c to 0x12db321 2024-09-16T11:43:42.978 INFO:teuthology.orchestra.run.smithi070.stdout:906 trunc from 0x12db321 to 0xde062e8 2024-09-16T11:43:42.984 INFO:teuthology.orchestra.run.smithi070.stdout:907 write 0x9d25a84 thru 0x9d290d5 (0x3652 bytes) 2024-09-16T11:43:42.987 INFO:teuthology.orchestra.run.smithi070.stdout:908 write 0xc4f620d thru 0xc4fcad3 (0x68c7 bytes) 2024-09-16T11:43:42.990 INFO:teuthology.orchestra.run.smithi070.stdout:909 punch from 0x696bdec to 0x696c9c1, (0xbd5 bytes) 2024-09-16T11:43:42.990 INFO:teuthology.orchestra.run.smithi070.stdout:910 write 0xc92a537 thru 0xc932f2a (0x89f4 bytes) 2024-09-16T11:43:42.993 INFO:teuthology.orchestra.run.smithi070.stdout:911 write 0x474145b thru 0x4743fd7 (0x2b7d bytes) 2024-09-16T11:43:42.995 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:42.995 INFO:teuthology.orchestra.run.smithi070.stdout:913 read 0x350a6e thru 0x3575a4 (0x6b37 bytes) 2024-09-16T11:43:42.996 INFO:teuthology.orchestra.run.smithi070.stdout:914 writesame 0x58704c4 thru 0x5876310 data_size 0xd(0x5e4d bytes) 2024-09-16T11:43:43.000 INFO:teuthology.orchestra.run.smithi070.stdout:916 punch from 0xa1b0df9 to 0xa1b4d9e, (0x3fa5 bytes) 2024-09-16T11:43:43.002 INFO:teuthology.orchestra.run.smithi070.stdout:917 trunc from 0xde062e8 to 0xbf2d7bb 2024-09-16T11:43:43.009 INFO:teuthology.orchestra.run.smithi070.stdout:918 writesame 0x45d4473 thru 0x45e43a3 data_size 0x1(0xff31 bytes) 2024-09-16T11:43:43.017 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:43.015+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:43.030 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:43.031 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:43.031 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:43:43.031 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:43.040 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:43:43.050 INFO:teuthology.orchestra.run.smithi070.stdout:919 trunc from 0xbf2d7bb to 0x3406807 2024-09-16T11:43:43.070 INFO:teuthology.orchestra.run.smithi070.stdout:920 write 0x4a1beb thru 0x4a1f2c (0x342 bytes) 2024-09-16T11:43:43.070 INFO:teuthology.orchestra.run.smithi070.stdout:921 read 0x90d75d thru 0x90dfe9 (0x88d bytes) 2024-09-16T11:43:43.071 INFO:teuthology.orchestra.run.smithi070.stdout:923 read 0x1210369 thru 0x12136a8 (0x3340 bytes) 2024-09-16T11:43:43.091 INFO:teuthology.orchestra.run.smithi070.stdout:925 write 0xde9a548 thru 0xde9b29a (0xd53 bytes) 2024-09-16T11:43:43.094 INFO:teuthology.orchestra.run.smithi070.stdout:926 read 0x2d81495 thru 0x2d87fe9 (0x6b55 bytes) 2024-09-16T11:43:43.095 INFO:teuthology.orchestra.run.smithi070.stdout:929 write 0xd470d52 thru 0xd47e83e (0xdaed bytes) 2024-09-16T11:43:43.098 INFO:teuthology.orchestra.run.smithi070.stdout:931 punch from 0x5df6f10 to 0x5e009b3, (0x9aa3 bytes) 2024-09-16T11:43:43.098 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.098 INFO:teuthology.orchestra.run.smithi070.stdout:933 write 0xf001eb thru 0xf06bd9 (0x69ef bytes) 2024-09-16T11:43:43.099 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.099 INFO:teuthology.orchestra.run.smithi070.stdout:937 punch from 0xd41a35d to 0xd429892, (0xf535 bytes) 2024-09-16T11:43:43.099 INFO:teuthology.orchestra.run.smithi070.stdout:941 writesame 0x9e7199d thru 0x9e7781a data_size 0x52(0x5e7e bytes) 2024-09-16T11:43:43.102 INFO:teuthology.orchestra.run.smithi070.stdout:942 read 0x7bf26b8 thru 0x7bfcdd0 (0xa719 bytes) 2024-09-16T11:43:43.102 INFO:teuthology.orchestra.run.smithi070.stdout:943 write 0x12f45ca thru 0x12f4f8c (0x9c3 bytes) 2024-09-16T11:43:43.103 INFO:teuthology.orchestra.run.smithi070.stdout:944 write 0xb27ba0 thru 0xb2e724 (0x6b85 bytes) 2024-09-16T11:43:43.104 INFO:teuthology.orchestra.run.smithi070.stdout:947 punch from 0x7951845 to 0x795947f, (0x7c3a bytes) 2024-09-16T11:43:43.104 INFO:teuthology.orchestra.run.smithi070.stdout:948 read 0x3966ddb thru 0x396b5cc (0x47f2 bytes) 2024-09-16T11:43:43.117 INFO:teuthology.orchestra.run.smithi070.stdout:950 trunc from 0xde9b29b to 0x36fdc7f 2024-09-16T11:43:43.124 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:43.123+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:43.128 INFO:teuthology.orchestra.run.smithi070.stdout:951 write 0x2f4c550 thru 0x2f4e871 (0x2322 bytes) 2024-09-16T11:43:43.132 INFO:teuthology.orchestra.run.smithi070.stdout:952 trunc from 0x36fdc7f to 0x52a27f2 2024-09-16T11:43:43.137 INFO:teuthology.orchestra.run.smithi070.stdout:953 read 0x6c9580 thru 0x6d0cad (0x772e bytes) 2024-09-16T11:43:43.138 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.138 INFO:teuthology.orchestra.run.smithi070.stdout:955 punch from 0x27b598e to 0x27c5886, (0xfef8 bytes) 2024-09-16T11:43:43.138 INFO:teuthology.orchestra.run.smithi070.stdout:956 punch from 0x5251999 to 0x525c924, (0xaf8b bytes) 2024-09-16T11:43:43.138 INFO:teuthology.orchestra.run.smithi070.stdout:957 punch from 0x3cbde7c to 0x3cc6b43, (0x8cc7 bytes) 2024-09-16T11:43:43.138 INFO:teuthology.orchestra.run.smithi070.stdout:958 read 0x46c2f92 thru 0x46ca4ee (0x755d bytes) 2024-09-16T11:43:43.149 INFO:teuthology.orchestra.run.smithi070.stdout:960 write 0xa8226ef thru 0xa82f9d0 (0xd2e2 bytes) 2024-09-16T11:43:43.152 INFO:teuthology.orchestra.run.smithi070.stdout:961 write 0x1aeaf3d thru 0x1afa65f (0xf723 bytes) 2024-09-16T11:43:43.154 INFO:teuthology.orchestra.run.smithi070.stdout:962 read 0x43361a5 thru 0x433c69e (0x64fa bytes) 2024-09-16T11:43:43.154 INFO:teuthology.orchestra.run.smithi070.stdout:963 read 0x809d34d thru 0x80ab734 (0xe3e8 bytes) 2024-09-16T11:43:43.154 INFO:teuthology.orchestra.run.smithi070.stdout:964 write 0x7ebc339 thru 0x7ebcf7f (0xc47 bytes) 2024-09-16T11:43:43.156 INFO:teuthology.orchestra.run.smithi070.stdout:965 punch from 0x87c48ac to 0x87c8288, (0x39dc bytes) 2024-09-16T11:43:43.156 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.157 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.161 INFO:teuthology.orchestra.run.smithi070.stdout:970 write 0xafb88e7 thru 0xafc5e81 (0xd59b bytes) 2024-09-16T11:43:43.165 INFO:teuthology.orchestra.run.smithi070.stdout:973 write 0x1b243ea thru 0x1b33a74 (0xf68b bytes) 2024-09-16T11:43:43.166 INFO:teuthology.orchestra.run.smithi070.stdout:974 punch from 0xa1a9666 to 0xa1b4944, (0xb2de bytes) 2024-09-16T11:43:43.174 INFO:teuthology.orchestra.run.smithi070.stdout:975 trunc from 0xafc5e82 to 0x433a7ab 2024-09-16T11:43:43.183 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.188 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:43.188 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:43.188 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:43.188 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:43.188 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:43.193 INFO:teuthology.orchestra.run.smithi070.stdout:977 write 0x7e1d40b thru 0x7e2aebe (0xdab4 bytes) 2024-09-16T11:43:43.197 INFO:teuthology.orchestra.run.smithi070.stdout:978 punch from 0x3b155b6 to 0x3b24f16, (0xf960 bytes) 2024-09-16T11:43:43.197 INFO:teuthology.orchestra.run.smithi070.stdout:980 read 0x3e25213 thru 0x3e3072d (0xb51b bytes) 2024-09-16T11:43:43.197 INFO:teuthology.orchestra.run.smithi070.stdout:981 read 0x1ab0bf2 thru 0x1abcc4f (0xc05e bytes) 2024-09-16T11:43:43.197 INFO:teuthology.orchestra.run.smithi070.stdout:982 read 0x68fb43a thru 0x6908883 (0xd44a bytes) 2024-09-16T11:43:43.198 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:43:43.210 INFO:teuthology.orchestra.run.smithi070.stdout:983 write 0xe7a6f5c thru 0xe7b506d (0xe112 bytes) 2024-09-16T11:43:43.218 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:43.216+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:43.229 INFO:teuthology.orchestra.run.smithi070.stdout:984 trunc from 0xe7b506e to 0xf835fe 2024-09-16T11:43:43.240 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.240 INFO:teuthology.orchestra.run.smithi070.stdout:986 read 0xd5817d thru 0xd64755 (0xc5d9 bytes) 2024-09-16T11:43:43.256 INFO:teuthology.orchestra.run.smithi070.stdout:987 write 0x9433293 thru 0x943dc42 (0xa9b0 bytes) 2024-09-16T11:43:43.259 INFO:teuthology.orchestra.run.smithi070.stdout:988 punch from 0x1694804 to 0x169ea87, (0xa283 bytes) 2024-09-16T11:43:43.259 INFO:teuthology.orchestra.run.smithi070.stdout:991 write 0x439fd34 thru 0x43a1b26 (0x1df3 bytes) 2024-09-16T11:43:43.261 INFO:teuthology.orchestra.run.smithi070.stdout:992 punch from 0x1210fe5 to 0x1212397, (0x13b2 bytes) 2024-09-16T11:43:43.267 INFO:teuthology.orchestra.run.smithi070.stdout:993 write 0xa641a68 thru 0xa6499f7 (0x7f90 bytes) 2024-09-16T11:43:43.270 INFO:teuthology.orchestra.run.smithi070.stdout:994 writesame 0x5b108c7 thru 0x5b126d5 data_size 0x39(0x1e0f bytes) 2024-09-16T11:43:43.273 INFO:teuthology.orchestra.run.smithi070.stdout:996 read 0x334e931 thru 0x3356cae (0x837e bytes) 2024-09-16T11:43:43.273 INFO:teuthology.orchestra.run.smithi070.stdout:997 read 0x1e30b40 thru 0x1e3119e (0x65f bytes) 2024-09-16T11:43:43.274 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.274 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.274 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.274 INFO:teuthology.orchestra.run.smithi070.stdout:1001 writesame 0x75b84e5 thru 0x75c63b0 data_size 0x62(0xdecc bytes) 2024-09-16T11:43:43.278 INFO:teuthology.orchestra.run.smithi070.stdout:1003 read 0x6a8a4dd thru 0x6a90e28 (0x694c bytes) 2024-09-16T11:43:43.278 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.278 INFO:teuthology.orchestra.run.smithi070.stdout:1005 write 0x773b389 thru 0x773c55c (0x11d4 bytes) 2024-09-16T11:43:43.280 INFO:teuthology.orchestra.run.smithi070.stdout:1009 punch from 0x4c3e82a to 0x4c42d0c, (0x44e2 bytes) 2024-09-16T11:43:43.280 INFO:teuthology.orchestra.run.smithi070.stdout:1010 write 0x31f090a thru 0x3200549 (0xfc40 bytes) 2024-09-16T11:43:43.282 INFO:teuthology.orchestra.run.smithi070.stdout:1011 punch from 0xe515a3 to 0xe5b5f1, (0xa04e bytes) 2024-09-16T11:43:43.283 INFO:teuthology.orchestra.run.smithi070.stdout:1013 trunc from 0xa6499f8 to 0x92dc573 2024-09-16T11:43:43.290 INFO:teuthology.orchestra.run.smithi070.stdout:1014 read 0x82c8108 thru 0x82d2b59 (0xaa52 bytes) 2024-09-16T11:43:43.302 INFO:teuthology.orchestra.run.smithi070.stdout:1015 write 0xebd9d1e thru 0xebe639a (0xc67d bytes) 2024-09-16T11:43:43.305 INFO:teuthology.orchestra.run.smithi070.stdout:1016 read 0xbe4f184 thru 0xbe5e367 (0xf1e4 bytes) 2024-09-16T11:43:43.305 INFO:teuthology.orchestra.run.smithi070.stdout:1018 read 0xeb370a0 thru 0xeb3f617 (0x8578 bytes) 2024-09-16T11:43:43.305 INFO:teuthology.orchestra.run.smithi070.stdout:1019 writesame 0xe4a6770 thru 0xe4a8f4b data_size 0x2(0x27dc bytes) 2024-09-16T11:43:43.310 INFO:teuthology.orchestra.run.smithi070.stdout:1020 punch from 0x61fc3c9 to 0x6203f11, (0x7b48 bytes) 2024-09-16T11:43:43.310 INFO:teuthology.orchestra.run.smithi070.stdout:1022 read 0x6ff1ab9 thru 0x6fff365 (0xd8ad bytes) 2024-09-16T11:43:43.310 INFO:teuthology.orchestra.run.smithi070.stdout:1023 punch from 0xb4e57d7 to 0xb4e68c5, (0x10ee bytes) 2024-09-16T11:43:43.317 INFO:teuthology.orchestra.run.smithi070.stdout:1024 trunc from 0xebe639b to 0x866cc02 2024-09-16T11:43:43.319 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:43.318+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:43.327 INFO:teuthology.orchestra.run.smithi070.stdout:1026 read 0x8af287 thru 0x8bca6a (0xd7e4 bytes) 2024-09-16T11:43:43.333 INFO:teuthology.orchestra.run.smithi070.stdout:1027 writesame 0x8e13877 thru 0x8e23206 data_size 0xb0(0xf990 bytes) 2024-09-16T11:43:43.334 INFO:teuthology.orchestra.run.smithi070.stdout:1030 write 0x1f6c088 thru 0x1f76da4 (0xad1d bytes) 2024-09-16T11:43:43.336 INFO:teuthology.orchestra.run.smithi070.stdout:1031 read 0x4b14956 thru 0x4b1b305 (0x69b0 bytes) 2024-09-16T11:43:43.343 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:43.343 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:43:43.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:43:43.344 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:43:43.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.344 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(4.18 epoch 34/28 rep_tid 399 entries 34'3 (0'0) error 4:1b51868d:::rbd_data.11c130c94979.0000000000000008:head by client.4545.0:1758 0.000000 -2 [r=-2+0b,r=0+0b] ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 31'1)", 2024-09-16T11:43:43.344 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.858107+0000", 2024-09-16T11:43:43.344 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.482682015, 2024-09-16T11:43:43.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0055608790000000003, 2024-09-16T11:43:43.344 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.344 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-16T11:43:43.344 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858107+0000", 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858107+0000", 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858107+0000", 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858112+0000", 2024-09-16T11:43:43.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1630000000000004e-06 2024-09-16T11:43:43.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858123+0000", 2024-09-16T11:43:43.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0519e-05 2024-09-16T11:43:43.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858129+0000", 2024-09-16T11:43:43.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6440000000000003e-06 2024-09-16T11:43:43.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858258+0000", 2024-09-16T11:43:43.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012866700000000001 2024-09-16T11:43:43.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.863667+0000", 2024-09-16T11:43:43.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054095360000000004 2024-09-16T11:43:43.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:2409 4.13 e34/28 4:cba8aa1e:::rbd_data.11c130c94979.0000000000000002:head v 34'176, mlcod=34'172)", 2024-09-16T11:43:43.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.899633+0000", 2024-09-16T11:43:43.348 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.441155663, 2024-09-16T11:43:43.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00513541, 2024-09-16T11:43:43.348 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.348 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.349 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.899633+0000", 2024-09-16T11:43:43.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.899633+0000", 2024-09-16T11:43:43.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.899634+0000", 2024-09-16T11:43:43.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4799999999999997e-07 2024-09-16T11:43:43.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.899643+0000", 2024-09-16T11:43:43.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7119999999999995e-06 2024-09-16T11:43:43.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.899761+0000", 2024-09-16T11:43:43.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117769 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.899767+0000", 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0009999999999998e-06 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.899802+0000", 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5166999999999997e-05 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.899823+0000", 2024-09-16T11:43:43.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0565e-05 2024-09-16T11:43:43.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.904685+0000", 2024-09-16T11:43:43.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048619450000000003 2024-09-16T11:43:43.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.904768+0000", 2024-09-16T11:43:43.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3403000000000001e-05 2024-09-16T11:43:43.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.352 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.353 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:2468 4.18 e34/28 4:1b51868d:::rbd_data.11c130c94979.0000000000000008:head v 34'4, mlcod=33'2)", 2024-09-16T11:43:43.353 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.900047+0000", 2024-09-16T11:43:43.353 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4407419600000004, 2024-09-16T11:43:43.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052454629999999997, 2024-09-16T11:43:43.353 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.353 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.353 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.900047+0000", 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.900047+0000", 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.900047+0000", 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8500000000000001e-07 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.900286+0000", 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000238796 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.900299+0000", 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3107e-05 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.900305+0000", 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9669999999999998e-06 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.900350+0000", 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4490999999999998e-05 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.900368+0000", 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-05 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.905265+0000", 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048972169999999997 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.905292+0000", 2024-09-16T11:43:43.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6820000000000001e-05 2024-09-16T11:43:43.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.357 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.357 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:2469 4.18 e34/28 4:1b51868d:::rbd_data.11c130c94979.0000000000000008:head v 34'6, mlcod=33'2)", 2024-09-16T11:43:43.357 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.901262+0000", 2024-09-16T11:43:43.357 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4395269219999998, 2024-09-16T11:43:43.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066226829999999999, 2024-09-16T11:43:43.357 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.357 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901262+0000", 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901262+0000", 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901262+0000", 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901270+0000", 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2520000000000002e-06 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901284+0000", 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.419e-05 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901289+0000", 2024-09-16T11:43:43.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:43:43.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901329+0000", 2024-09-16T11:43:43.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9666999999999998e-05 2024-09-16T11:43:43.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901347+0000", 2024-09-16T11:43:43.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8201000000000001e-05 2024-09-16T11:43:43.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.907850+0000", 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0065027949999999996 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.907884+0000", 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4261999999999998e-05 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2484 4.5 4:a6f13331:::rbd_data.11c130c94979.0000000000000018:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.912091+0000", 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4286977619999996, 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0074078199999999999, 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2484 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912091+0000", 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912091+0000", 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912097+0000", 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6420000000000001e-06 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912104+0000", 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.472e-06 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912122+0000", 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7728999999999999e-05 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912129+0000", 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1369999999999996e-06 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912182+0000", 2024-09-16T11:43:43.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2800000000000003e-05 2024-09-16T11:43:43.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912259+0000", 2024-09-16T11:43:43.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7628999999999998e-05 2024-09-16T11:43:43.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-16T11:43:43.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912446+0000", 2024-09-16T11:43:43.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018684600000000001 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.913585+0000", 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011388030000000001 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919426+0000", 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0058407909999999997 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919474+0000", 2024-09-16T11:43:43.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8044999999999998e-05 2024-09-16T11:43:43.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919499+0000", 2024-09-16T11:43:43.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4926000000000002e-05 2024-09-16T11:43:43.367 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.367 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.367 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.367 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2499 4.a 4:56ab1d25:::rbd_data.11c130c94979.0000000000000067:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:43.367 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.916473+0000", 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4243152549999998, 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051807570000000002, 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2499 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916473+0000", 2024-09-16T11:43:43.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916473+0000", 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916475+0000", 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.238e-06 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916479+0000", 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0080000000000004e-06 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916492+0000", 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2998e-05 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916498+0000", 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9900000000000002e-06 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916538+0000", 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0633999999999998e-05 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916625+0000", 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6878000000000004e-05 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916795+0000", 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000169977 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.918123+0000", 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013279019999999999 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.921583+0000", 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034603220000000001 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.921629+0000", 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5448999999999998e-05 2024-09-16T11:43:43.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.921654+0000", 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5361e-05 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2501 4.5 4:a3a65b20:::rbd_data.11c130c94979.000000000000008d:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.916515+0000", 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4242736090000001, 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054241840000000003, 2024-09-16T11:43:43.373 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2501 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916515+0000", 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916515+0000", 2024-09-16T11:43:43.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916515+0000", 2024-09-16T11:43:43.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0500000000000004e-07 2024-09-16T11:43:43.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916518+0000", 2024-09-16T11:43:43.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4889999999999998e-06 2024-09-16T11:43:43.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916525+0000", 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8759999999999999e-06 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916529+0000", 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3139999999999997e-06 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916570+0000", 2024-09-16T11:43:43.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1013999999999997e-05 2024-09-16T11:43:43.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916646+0000", 2024-09-16T11:43:43.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6200999999999998e-05 2024-09-16T11:43:43.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-16T11:43:43.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916740+0000", 2024-09-16T11:43:43.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3535000000000005e-05 2024-09-16T11:43:43.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.918799+0000", 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020588680000000002 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.921878+0000", 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030790359999999998 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.921917+0000", 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9684999999999997e-05 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.921939+0000", 2024-09-16T11:43:43.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1661000000000001e-05 2024-09-16T11:43:43.379 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.379 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.379 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.379 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:2504 4.18 e34/28 4:1cbc85dd:::rbd_data.11c130c94979.00000000000000d9:head v 34'8, mlcod=34'4)", 2024-09-16T11:43:43.379 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.918999+0000", 2024-09-16T11:43:43.379 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4217893650000004, 2024-09-16T11:43:43.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054815239999999998, 2024-09-16T11:43:43.379 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.380 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.380 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.918999+0000", 2024-09-16T11:43:43.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.918999+0000", 2024-09-16T11:43:43.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919000+0000", 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6199999999999997e-07 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919005+0000", 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4090000000000002e-06 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919017+0000", 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1960999999999999e-05 2024-09-16T11:43:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919024+0000", 2024-09-16T11:43:43.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4370000000000004e-06 2024-09-16T11:43:43.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919073+0000", 2024-09-16T11:43:43.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9638999999999997e-05 2024-09-16T11:43:43.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919092+0000", 2024-09-16T11:43:43.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8763999999999999e-05 2024-09-16T11:43:43.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.924429+0000", 2024-09-16T11:43:43.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053370099999999997 2024-09-16T11:43:43.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.924481+0000", 2024-09-16T11:43:43.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1641999999999998e-05 2024-09-16T11:43:43.383 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.383 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.384 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:5360 4.1f 4:f871e0e5:::rbd_data.11c18a2497a7.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:43.384 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.001916+0000", 2024-09-16T11:43:43.384 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.338872045, 2024-09-16T11:43:43.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0069369610000000002, 2024-09-16T11:43:43.384 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.384 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.384 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.384 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.384 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.384 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5360 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.001916+0000", 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.001916+0000", 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.001918+0000", 2024-09-16T11:43:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:43:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.001926+0000", 2024-09-16T11:43:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5279999999999998e-06 2024-09-16T11:43:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.001942+0000", 2024-09-16T11:43:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5954999999999999e-05 2024-09-16T11:43:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.001949+0000", 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.34e-06 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.002019+0000", 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9601e-05 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.002077+0000", 2024-09-16T11:43:43.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7868000000000002e-05 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.003231+0000", 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011540299999999999 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.008792+0000", 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0055614970000000003 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.008800+0000", 2024-09-16T11:43:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8339999999999999e-06 2024-09-16T11:43:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.008829+0000", 2024-09-16T11:43:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8982000000000001e-05 2024-09-16T11:43:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.008853+0000", 2024-09-16T11:43:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4592000000000001e-05 2024-09-16T11:43:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.389 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:5469 4.1f 4:fd754f81:::rbd_data.11c18a2497a7.0000000000000006:head [stat,truncate 3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.143932+0000", 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.196856071, 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051220559999999998, 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5469 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.390 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.143932+0000", 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.143932+0000", 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.143934+0000", 2024-09-16T11:43:43.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.08e-06 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.143938+0000", 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9099999999999996e-06 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.143949+0000", 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0266e-05 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.143956+0000", 2024-09-16T11:43:43.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9589999999999998e-06 2024-09-16T11:43:43.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144003+0000", 2024-09-16T11:43:43.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6990000000000002e-05 2024-09-16T11:43:43.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144085+0000", 2024-09-16T11:43:43.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1880999999999998e-05 2024-09-16T11:43:43.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144203+0000", 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011866600000000001 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.146074+0000", 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018705250000000001 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.148973+0000", 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028991759999999998 2024-09-16T11:43:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.149017+0000", 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3605e-05 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.149055+0000", 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7997999999999998e-05 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:5474 4.1f 4:fc0e8e36:::rbd_data.11c18a2497a7.000000000000000b:head [stat,truncate 3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.144003+0000", 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.1967852960000001, 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052033560000000001, 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5474 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144003+0000", 2024-09-16T11:43:43.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144003+0000", 2024-09-16T11:43:43.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144004+0000", 2024-09-16T11:43:43.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:43:43.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144007+0000", 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.377e-06 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144016+0000", 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8740000000000001e-06 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144021+0000", 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5619999999999997e-06 2024-09-16T11:43:43.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144454+0000", 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00043323899999999998 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144522+0000", 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8251999999999996e-05 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144642+0000", 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000119677 2024-09-16T11:43:43.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.149095+0000", 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044535379999999999 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.149130+0000", 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999997e-05 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.149171+0000", 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.019e-05 2024-09-16T11:43:43.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.149207+0000", 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5997000000000001e-05 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:5476 4.1f 4:f871e0e5:::rbd_data.11c18a2497a7.000000000000000d:head [stat,truncate 3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.144043+0000", 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.1967452869999999, 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052398239999999997, 2024-09-16T11:43:43.401 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5476 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144043+0000", 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144043+0000", 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144044+0000", 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7300000000000001e-07 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144046+0000", 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144053+0000", 2024-09-16T11:43:43.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5440000000000004e-06 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.144056+0000", 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8010000000000001e-06 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.145023+0000", 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00096679800000000005 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.145105+0000", 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1407000000000002e-05 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.145199+0000", 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4209999999999994e-05 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.149124+0000", 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039254010000000002 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.149214+0000", 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9291999999999998e-05 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.149248+0000", 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4672999999999998e-05 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.149283+0000", 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4780000000000002e-05 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.407 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:5665 4.1f 4:fc0e8e36:::rbd_data.11c18a2497a7.000000000000000b:head [truncate 3670016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:43.407 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.361988+0000", 2024-09-16T11:43:43.407 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.97880093700000004, 2024-09-16T11:43:43.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062122430000000001, 2024-09-16T11:43:43.407 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.407 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.407 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5665 2024-09-16T11:43:43.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.407 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.361988+0000", 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.361988+0000", 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.361988+0000", 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4600000000000005e-07 2024-09-16T11:43:43.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.361991+0000", 2024-09-16T11:43:43.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5890000000000001e-06 2024-09-16T11:43:43.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.361998+0000", 2024-09-16T11:43:43.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3599999999999998e-06 2024-09-16T11:43:43.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362003+0000", 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.425e-06 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362369+0000", 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00036655599999999998 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362438+0000", 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9141999999999996e-05 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362768+0000", 2024-09-16T11:43:43.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032945899999999999 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.367897+0000", 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051293060000000001 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.368160+0000", 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026256899999999999 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.368181+0000", 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1297999999999998e-05 2024-09-16T11:43:43.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.368200+0000", 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8992999999999999e-05 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:5667 4.1f 4:f871e0e5:::rbd_data.11c18a2497a7.000000000000000d:head [truncate 3670016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.362010+0000", 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.97877882199999999, 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062621719999999999, 2024-09-16T11:43:43.412 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5667 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362010+0000", 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362010+0000", 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362010+0000", 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362012+0000", 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9829999999999999e-06 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362018+0000", 2024-09-16T11:43:43.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1569999999999997e-06 2024-09-16T11:43:43.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362022+0000", 2024-09-16T11:43:43.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7230000000000002e-06 2024-09-16T11:43:43.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362971+0000", 2024-09-16T11:43:43.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00094911799999999999 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.363056+0000", 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5334999999999995e-05 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.363167+0000", 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000110214 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.367906+0000", 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047389420000000003 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.368227+0000", 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032165999999999998 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.368253+0000", 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5984000000000001e-05 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.368272+0000", 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8620000000000001e-05 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6011 4.1f 4:f871e0e5:::rbd_data.11c18a2497a7.000000000000000d:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.782009+0000", 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.558779621, 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.006894753, 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6011 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.782009+0000", 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.782009+0000", 2024-09-16T11:43:43.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.782009+0000", 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.89e-07 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.782011+0000", 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.747e-06 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.782016+0000", 2024-09-16T11:43:43.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3020000000000002e-06 2024-09-16T11:43:43.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.782020+0000", 2024-09-16T11:43:43.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4580000000000002e-06 2024-09-16T11:43:43.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.783171+0000", 2024-09-16T11:43:43.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001151551 2024-09-16T11:43:43.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.783260+0000", 2024-09-16T11:43:43.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8172999999999998e-05 2024-09-16T11:43:43.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:43.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.783368+0000", 2024-09-16T11:43:43.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000108356 2024-09-16T11:43:43.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.786458+0000", 2024-09-16T11:43:43.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003089854 2024-09-16T11:43:43.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.788847+0000", 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023892420000000002 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.788874+0000", 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6784999999999999e-05 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.788904+0000", 2024-09-16T11:43:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9896e-05 2024-09-16T11:43:43.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.424 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.424 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6160 4.1f 4:fc0e8e36:::rbd_data.11c18a2497a7.000000000000000b:head [truncate 1310720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:43.424 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.950545+0000", 2024-09-16T11:43:43.424 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.39024302700000002, 2024-09-16T11:43:43.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062640730000000002, 2024-09-16T11:43:43.424 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.424 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6160 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950545+0000", 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950545+0000", 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950546+0000", 2024-09-16T11:43:43.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2099999999999997e-07 2024-09-16T11:43:43.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950549+0000", 2024-09-16T11:43:43.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.599e-06 2024-09-16T11:43:43.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950555+0000", 2024-09-16T11:43:43.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1460000000000001e-06 2024-09-16T11:43:43.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950559+0000", 2024-09-16T11:43:43.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4320000000000001e-06 2024-09-16T11:43:43.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.951037+0000", 2024-09-16T11:43:43.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047762199999999997 2024-09-16T11:43:43.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.951105+0000", 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7830999999999995e-05 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.951213+0000", 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000108262 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.953849+0000", 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026364439999999999 2024-09-16T11:43:43.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956751+0000", 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0029016020000000001 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956786+0000", 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4904999999999999e-05 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956810+0000", 2024-09-16T11:43:43.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3708999999999999e-05 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6162 4.1f 4:f871e0e5:::rbd_data.11c18a2497a7.000000000000000d:head [truncate 1310720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.950591+0000", 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.39019774200000001, 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.006297268, 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6162 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950591+0000", 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950591+0000", 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950592+0000", 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5900000000000005e-07 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950596+0000", 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7350000000000004e-06 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950609+0000", 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2656e-05 2024-09-16T11:43:43.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950616+0000", 2024-09-16T11:43:43.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9319999999999999e-06 2024-09-16T11:43:43.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.951460+0000", 2024-09-16T11:43:43.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00084344300000000004 2024-09-16T11:43:43.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.951535+0000", 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5768999999999999e-05 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.951646+0000", 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011105 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956549+0000", 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049029329999999999 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956842+0000", 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029260899999999999 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956869+0000", 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7010000000000001e-05 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956888+0000", 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9171999999999999e-05 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.435 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6219 4.1f 4:fd754f81:::rbd_data.11c18a2497a7.0000000000000006:head [truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:43.056806+0000", 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.28398259100000001, 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0056997819999999996, 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6219 2024-09-16T11:43:43.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056806+0000", 2024-09-16T11:43:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056806+0000", 2024-09-16T11:43:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056807+0000", 2024-09-16T11:43:43.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7000000000000003e-07 2024-09-16T11:43:43.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056811+0000", 2024-09-16T11:43:43.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3900000000000003e-06 2024-09-16T11:43:43.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056823+0000", 2024-09-16T11:43:43.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1945e-05 2024-09-16T11:43:43.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056829+0000", 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8529999999999997e-06 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056881+0000", 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1425000000000003e-05 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056967+0000", 2024-09-16T11:43:43.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6638999999999995e-05 2024-09-16T11:43:43.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:43.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057074+0000", 2024-09-16T11:43:43.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010676800000000001 2024-09-16T11:43:43.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.059622+0000", 2024-09-16T11:43:43.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025480659999999999 2024-09-16T11:43:43.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062457+0000", 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028349759999999999 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062484+0000", 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6950000000000001e-05 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062506+0000", 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1800000000000001e-05 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.441 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6224 4.1f 4:fc0e8e36:::rbd_data.11c18a2497a7.000000000000000b:head [truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:43.056842+0000", 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.28394698400000001, 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057434119999999998, 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6224 2024-09-16T11:43:43.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056842+0000", 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056842+0000", 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056842+0000", 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2699999999999999e-07 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056847+0000", 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4619999999999998e-06 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056854+0000", 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056859+0000", 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5419999999999996e-06 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057350+0000", 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00049094600000000003 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057407+0000", 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6375999999999997e-05 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057508+0000", 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101265 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062398+0000", 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048902570000000003 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062538+0000", 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140016 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062562+0000", 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4417999999999999e-05 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062585+0000", 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2486000000000001e-05 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6226 4.1f 4:f871e0e5:::rbd_data.11c18a2497a7.000000000000000d:head [truncate 3670016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:43.056935+0000", 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.28385331600000002, 2024-09-16T11:43:43.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057163179999999997, 2024-09-16T11:43:43.448 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:43.448 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:43.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:43.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:43.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:43.448 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6226 2024-09-16T11:43:43.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.449 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:43.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:43.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056935+0000", 2024-09-16T11:43:43.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:43.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056935+0000", 2024-09-16T11:43:43.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:43.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056936+0000", 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8299999999999995e-07 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056941+0000", 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7759999999999997e-06 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056954+0000", 2024-09-16T11:43:43.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2858e-05 2024-09-16T11:43:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056960+0000", 2024-09-16T11:43:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.959e-06 2024-09-16T11:43:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057694+0000", 2024-09-16T11:43:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00073463999999999995 2024-09-16T11:43:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057767+0000", 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2349999999999997e-05 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057878+0000", 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000111365 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062404+0000", 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045255590000000002 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:43.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062611+0000", 2024-09-16T11:43:43.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020681399999999999 2024-09-16T11:43:43.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:43.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062632+0000", 2024-09-16T11:43:43.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1469999999999999e-05 2024-09-16T11:43:43.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:43.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:43.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:43.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062652+0000", 2024-09-16T11:43:43.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9544000000000001e-05 2024-09-16T11:43:43.454 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.454 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.454 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.454 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:43.454 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:43.454 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:43.456 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:43:43.457 INFO:teuthology.orchestra.run.smithi070.stdout:1032 write 0xe5227e4 thru 0xe5309f1 (0xe20e bytes) 2024-09-16T11:43:43.457 INFO:teuthology.orchestra.run.smithi070.stdout:1033 punch from 0x12b4dcb to 0x12b67e5, (0x1a1a bytes) 2024-09-16T11:43:43.458 INFO:teuthology.orchestra.run.smithi070.stdout:1037 trunc from 0xe5309f2 to 0xb02dce6 2024-09-16T11:43:43.458 INFO:teuthology.orchestra.run.smithi070.stdout:truncating to largest ever: 0xed8811e 2024-09-16T11:43:43.458 INFO:teuthology.orchestra.run.smithi070.stdout:1038 trunc from 0xb02dce6 to 0xed8811e 2024-09-16T11:43:43.458 INFO:teuthology.orchestra.run.smithi070.stdout:1039 read 0x7c30811 thru 0x7c3e895 (0xe085 bytes) 2024-09-16T11:43:43.458 INFO:teuthology.orchestra.run.smithi070.stdout:1041 writesame 0x2db016a thru 0x2db3e6d data_size 0x5(0x3d04 bytes) 2024-09-16T11:43:43.458 INFO:teuthology.orchestra.run.smithi070.stdout:1042 read 0x74b9fbb thru 0x74c7b51 (0xdb97 bytes) 2024-09-16T11:43:43.458 INFO:teuthology.orchestra.run.smithi070.stdout:1044 write 0xe5318ab thru 0xe53f363 (0xdab9 bytes) 2024-09-16T11:43:43.458 INFO:teuthology.orchestra.run.smithi070.stdout:1046 writesame 0xe4ff806 thru 0xe503b7c data_size 0x13(0x4377 bytes) 2024-09-16T11:43:43.458 INFO:teuthology.orchestra.run.smithi070.stdout:1047 read 0xe0dedbc thru 0xe0e6bbf (0x7e04 bytes) 2024-09-16T11:43:43.458 INFO:teuthology.orchestra.run.smithi070.stdout:1048 writesame 0x21e921f thru 0x21f54db data_size 0x1(0xc2bd bytes) 2024-09-16T11:43:43.458 INFO:teuthology.orchestra.run.smithi070.stdout:1049 write 0xe7b265d thru 0xe7b7084 (0x4a28 bytes) 2024-09-16T11:43:43.458 INFO:teuthology.orchestra.run.smithi070.stdout:1050 write 0xd2f5271 thru 0xd3036ec (0xe47c bytes) 2024-09-16T11:43:43.458 INFO:teuthology.orchestra.run.smithi070.stdout:1051 trunc from 0xed8811e to 0x87a9e31 2024-09-16T11:43:43.459 INFO:teuthology.orchestra.run.smithi070.stdout:1052 punch from 0x5aefbaf to 0x5afc001, (0xc452 bytes) 2024-09-16T11:43:43.459 INFO:teuthology.orchestra.run.smithi070.stdout:1053 read 0x7f68717 thru 0x7f6a817 (0x2101 bytes) 2024-09-16T11:43:43.459 INFO:teuthology.orchestra.run.smithi070.stdout:1054 trunc from 0x87a9e31 to 0xcde7e5 2024-09-16T11:43:43.459 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.459 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:43.453+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:43.463 INFO:teuthology.orchestra.run.smithi070.stdout:1057 write 0x41783c7 thru 0x41830d7 (0xad11 bytes) 2024-09-16T11:43:43.469 INFO:teuthology.orchestra.run.smithi070.stdout:1058 trunc from 0x41830d8 to 0x22378a7 2024-09-16T11:43:43.498 INFO:teuthology.orchestra.run.smithi070.stdout:1059 writesame 0x713071c thru 0x713d208 data_size 0x1(0xcaed bytes) 2024-09-16T11:43:43.498 INFO:teuthology.orchestra.run.smithi070.stdout:1060 trunc from 0x713d209 to 0x70bd646 2024-09-16T11:43:43.501 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.502 INFO:teuthology.orchestra.run.smithi070.stdout:1062 read 0x1cf16d3 thru 0x1cf3802 (0x2130 bytes) 2024-09-16T11:43:43.502 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.502 INFO:teuthology.orchestra.run.smithi070.stdout:1064 read 0x2011da6 thru 0x20186bd (0x6918 bytes) 2024-09-16T11:43:43.514 INFO:teuthology.orchestra.run.smithi070.stdout:1065 write 0xc976b4e thru 0xc985c9f (0xf152 bytes) 2024-09-16T11:43:43.517 INFO:teuthology.orchestra.run.smithi070.stdout:1066 write 0x83159e8 thru 0x831b3ea (0x5a03 bytes) 2024-09-16T11:43:43.521 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.521 INFO:teuthology.orchestra.run.smithi070.stdout:1069 write 0x8ec81eb thru 0x8ecfd61 (0x7b77 bytes) 2024-09-16T11:43:43.523 INFO:teuthology.orchestra.run.smithi070.stdout:1070 write 0x7c2b058 thru 0x7c38039 (0xcfe2 bytes) 2024-09-16T11:43:43.525 INFO:teuthology.orchestra.run.smithi070.stdout:1071 writesame 0x34a31e4 thru 0x34a7a8c data_size 0x13(0x48a9 bytes) 2024-09-16T11:43:43.528 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.528 INFO:teuthology.orchestra.run.smithi070.stdout:1073 write 0xc5da6c0 thru 0xc5dad72 (0x6b3 bytes) 2024-09-16T11:43:43.529 INFO:teuthology.orchestra.run.smithi070.stdout:1074 write 0x81762b4 thru 0x81772c5 (0x1012 bytes) 2024-09-16T11:43:43.530 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:43:43.531 INFO:teuthology.orchestra.run.smithi070.stdout:1075 read 0x92499a6 thru 0x92499c2 (0x1d bytes) 2024-09-16T11:43:43.532 INFO:teuthology.orchestra.run.smithi070.stdout:1076 read 0x5a8c7e3 thru 0x5a90588 (0x3da6 bytes) 2024-09-16T11:43:43.532 INFO:teuthology.orchestra.run.smithi070.stdout:1078 read 0x985c923 thru 0x985e788 (0x1e66 bytes) 2024-09-16T11:43:43.532 INFO:teuthology.orchestra.run.smithi070.stdout:1079 read 0x580f213 thru 0x5814a93 (0x5881 bytes) 2024-09-16T11:43:43.532 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.532 INFO:teuthology.orchestra.run.smithi070.stdout:1082 write 0x74634b7 thru 0x7469951 (0x649b bytes) 2024-09-16T11:43:43.534 INFO:teuthology.orchestra.run.smithi070.stdout:1083 read 0xc703c1c thru 0xc70dea7 (0xa28c bytes) 2024-09-16T11:43:43.534 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.534 INFO:teuthology.orchestra.run.smithi070.stdout:1085 punch from 0xab11ef8 to 0xab154bf, (0x35c7 bytes) 2024-09-16T11:43:43.534 INFO:teuthology.orchestra.run.smithi070.stdout:1086 read 0x8bf880f thru 0x8c0289d (0xa08f bytes) 2024-09-16T11:43:43.540 INFO:teuthology.orchestra.run.smithi070.stdout:1088 trunc from 0xc985ca0 to 0x7a4794d 2024-09-16T11:43:43.557 INFO:teuthology.orchestra.run.smithi070.stdout:1089 write 0x91d09a4 thru 0x91dfdfa (0xf457 bytes) 2024-09-16T11:43:43.557 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:43.555+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:43.565 INFO:teuthology.orchestra.run.smithi070.stdout:1090 trunc from 0x91dfdfb to 0x3c5872a 2024-09-16T11:43:43.583 INFO:teuthology.orchestra.run.smithi070.stdout:1091 trunc from 0x3c5872a to 0xa9b4d27 2024-09-16T11:43:43.588 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.590 INFO:teuthology.orchestra.run.smithi070.stdout:1096 trunc from 0xa9b4d27 to 0x93947c9 2024-09-16T11:43:43.595 INFO:teuthology.orchestra.run.smithi070.stdout:1098 trunc from 0x93947c9 to 0xa1ee683 2024-09-16T11:43:43.599 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:43.599 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:43.599 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:43:43.599 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:43.600 INFO:teuthology.orchestra.run.smithi070.stdout:1099 writesame 0x2e07043 thru 0x2e0d9cf data_size 0x1(0x698d bytes) 2024-09-16T11:43:43.609 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:43:43.612 INFO:teuthology.orchestra.run.smithi070.stdout:1100 read 0x9efe283 thru 0x9f06dc9 (0x8b47 bytes) 2024-09-16T11:43:43.612 INFO:teuthology.orchestra.run.smithi070.stdout:1101 read 0x89a6ebc thru 0x89ad4cb (0x6610 bytes) 2024-09-16T11:43:43.620 INFO:teuthology.orchestra.run.smithi070.stdout:1103 write 0xd121e1c thru 0xd12bca3 (0x9e88 bytes) 2024-09-16T11:43:43.623 INFO:teuthology.orchestra.run.smithi070.stdout:1104 write 0xaa749c3 thru 0xaa8011e (0xb75c bytes) 2024-09-16T11:43:43.631 INFO:teuthology.orchestra.run.smithi070.stdout:1105 trunc from 0xd12bca4 to 0x8cbee8b 2024-09-16T11:43:43.639 INFO:teuthology.orchestra.run.smithi070.stdout:1106 read 0xae59a3 thru 0xaecfdc (0x763a bytes) 2024-09-16T11:43:43.644 INFO:teuthology.orchestra.run.smithi070.stdout:1107 trunc from 0x8cbee8b to 0x5621a2c 2024-09-16T11:43:43.657 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:43:43.658 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:43:43.662 INFO:teuthology.orchestra.run.smithi070.stdout:1108 writesame 0xcd29de4 thru 0xcd33775 data_size 0xb(0x9992 bytes) 2024-09-16T11:43:43.663 INFO:teuthology.orchestra.run.smithi070.stdout:1109 writesame 0x2af8940 thru 0x2b0083c data_size 0x1(0x7efd bytes) 2024-09-16T11:43:43.674 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:43:43.675 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:43:43.676 INFO:teuthology.orchestra.run.smithi070.stdout:1110 writesame 0x39d8291 thru 0x39de434 data_size 0x6(0x61a4 bytes) 2024-09-16T11:43:43.681 INFO:teuthology.orchestra.run.smithi070.stdout:1111 write 0x6fa0988 thru 0x6fa7b7e (0x71f7 bytes) 2024-09-16T11:43:43.683 INFO:teuthology.orchestra.run.smithi070.stdout:1113 writesame 0x159fe7b thru 0x15a0b6f data_size 0x1(0xcf5 bytes) 2024-09-16T11:43:43.686 INFO:teuthology.orchestra.run.smithi070.stdout:1114 read 0xc888998 thru 0xc88d94b (0x4fb4 bytes) 2024-09-16T11:43:43.686 INFO:teuthology.orchestra.run.smithi070.stdout:1115 read 0x2c0bf4f thru 0x2c1b62b (0xf6dd bytes) 2024-09-16T11:43:43.686 INFO:teuthology.orchestra.run.smithi070.stdout:1116 read 0x737bf5c thru 0x737cb3f (0xbe4 bytes) 2024-09-16T11:43:43.692 INFO:teuthology.orchestra.run.smithi070.stdout:1117 writesame 0xced2b9c thru 0xced43a5 data_size 0x11(0x180a bytes) 2024-09-16T11:43:43.692 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.692 INFO:teuthology.orchestra.run.smithi070.stdout:1120 writesame 0x3afeff2 thru 0x3b04c4b data_size 0x1(0x5c5a bytes) 2024-09-16T11:43:43.692 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:43:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:43:43.694 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:43.693+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:43.698 INFO:teuthology.orchestra.run.smithi070.stdout:1123 writesame 0xaf823aa thru 0xaf86660 data_size 0x1(0x42b7 bytes) 2024-09-16T11:43:43.708 INFO:teuthology.orchestra.run.smithi070.stdout:1124 read 0x6793b2d thru 0x67a1e0f (0xe2e3 bytes) 2024-09-16T11:43:43.708 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:43:43.708 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:43:43.714 INFO:teuthology.orchestra.run.smithi070.stdout:1125 write 0xe9f080a thru 0xe9f7434 (0x6c2b bytes) 2024-09-16T11:43:43.717 INFO:teuthology.orchestra.run.smithi070.stdout:1126 writesame 0xccaea22 thru 0xccbb2cd data_size 0x12(0xc8ac bytes) 2024-09-16T11:43:43.721 INFO:teuthology.orchestra.run.smithi070.stdout:1127 write 0xe5692db thru 0xe577624 (0xe34a bytes) 2024-09-16T11:43:43.724 INFO:teuthology.orchestra.run.smithi070.stdout:1128 punch from 0x36e3f80 to 0x36e8f45, (0x4fc5 bytes) 2024-09-16T11:43:43.724 INFO:teuthology.orchestra.run.smithi070.stdout:1129 read 0x12702f1 thru 0x1271cb4 (0x19c4 bytes) 2024-09-16T11:43:43.724 INFO:teuthology.orchestra.run.smithi070.stdout:1130 read 0x5cefd9a thru 0x5cf7221 (0x7488 bytes) 2024-09-16T11:43:43.725 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:43:43.726 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:43:43.726 INFO:teuthology.orchestra.run.smithi070.stdout:1132 read 0xb3da52d thru 0xb3e9962 (0xf436 bytes) 2024-09-16T11:43:43.726 INFO:teuthology.orchestra.run.smithi070.stdout:1133 write 0xd03e144 thru 0xd046056 (0x7f13 bytes) 2024-09-16T11:43:43.728 INFO:teuthology.orchestra.run.smithi070.stdout:1134 punch from 0x5c61572 to 0x5c65053, (0x3ae1 bytes) 2024-09-16T11:43:43.728 INFO:teuthology.orchestra.run.smithi070.stdout:1135 writesame 0x57f3f11 thru 0x57f3f5b data_size 0x1(0x4b bytes) 2024-09-16T11:43:43.730 INFO:teuthology.orchestra.run.smithi070.stdout:1136 write 0xafc48d1 thru 0xafd07d9 (0xbf09 bytes) 2024-09-16T11:43:43.733 INFO:teuthology.orchestra.run.smithi070.stdout:1137 read 0x184533e thru 0x185098c (0xb64f bytes) 2024-09-16T11:43:43.733 INFO:teuthology.orchestra.run.smithi070.stdout:1138 writesame 0x4bacafd thru 0x4bb68d2 data_size 0x2(0x9dd6 bytes) 2024-09-16T11:43:43.742 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:43:43.742 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:43:43.744 INFO:teuthology.orchestra.run.smithi070.stdout:1140 read 0xa9b6043 thru 0xa9b9451 (0x340f bytes) 2024-09-16T11:43:43.744 INFO:teuthology.orchestra.run.smithi070.stdout:1141 punch from 0x7a02d0e to 0x7a0d1de, (0xa4d0 bytes) 2024-09-16T11:43:43.744 INFO:teuthology.orchestra.run.smithi070.stdout:1142 writesame 0xe7b810e thru 0xe7be020 data_size 0x3d(0x5f13 bytes) 2024-09-16T11:43:43.747 INFO:teuthology.orchestra.run.smithi070.stdout:1143 punch from 0x7feaaa8 to 0x7feef0d, (0x4465 bytes) 2024-09-16T11:43:43.748 INFO:teuthology.orchestra.run.smithi070.stdout:1144 trunc from 0xe9f7435 to 0xdccb7d2 2024-09-16T11:43:43.753 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:43.753 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:43.758 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:43.757+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:43.759 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:43:43.759 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:43:43.760 INFO:teuthology.orchestra.run.smithi070.stdout:1145 write 0xa71501a thru 0xa724a43 (0xfa2a bytes) 2024-09-16T11:43:43.763 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:43:43.764 INFO:teuthology.orchestra.run.smithi070.stdout:1146 trunc from 0xdccb7d2 to 0xd6db8e9 2024-09-16T11:43:43.769 INFO:teuthology.orchestra.run.smithi070.stdout:1147 punch from 0xc61365a to 0xc6208d1, (0xd277 bytes) 2024-09-16T11:43:43.769 INFO:teuthology.orchestra.run.smithi070.stdout:1148 write 0x4b90f23 thru 0x4b943cc (0x34aa bytes) 2024-09-16T11:43:43.770 INFO:teuthology.orchestra.run.smithi070.stdout:1149 writesame 0x21a342a thru 0x21b0905 data_size 0x72(0xd4dc bytes) 2024-09-16T11:43:43.772 INFO:teuthology.orchestra.run.smithi070.stdout:1150 read 0x996b9a8 thru 0x996ee3b (0x3494 bytes) 2024-09-16T11:43:43.772 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:43.772 INFO:teuthology.orchestra.run.smithi070.stdout:1152 write 0x49d464 thru 0x4a029d (0x2e3a bytes) 2024-09-16T11:43:43.774 INFO:teuthology.orchestra.run.smithi070.stdout:1153 punch from 0x1aee411 to 0x1afe22c, (0xfe1b bytes) 2024-09-16T11:43:43.774 INFO:teuthology.orchestra.run.smithi070.stdout:1154 write 0x95bd6b5 thru 0x95c4441 (0x6d8d bytes) 2024-09-16T11:43:43.776 INFO:teuthology.orchestra.run.smithi070.stdout:1155 clone 7 order 23 su 131072 sc 9 2024-09-16T11:43:43.777 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:43:43.777 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:43:43.860 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:43.858+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:43.903 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:43.904 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:43.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:43:43.961 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:43.959+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:44.018 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:44.019 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:44.028 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:43:44.029 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-16T11:43:44.072 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:44.070+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:44.151 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:44.151 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:44.161 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:43:44.163 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:44.161+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:44.191 INFO:teuthology.orchestra.run.smithi070.stdout:1156 write 0xac002e0 thru 0xac03e5a (0x3b7b bytes) 2024-09-16T11:43:44.293 INFO:teuthology.orchestra.run.smithi070.stdout:1157 writesame 0xbcbd88f thru 0xbccc0fc data_size 0x6(0xe86e bytes) 2024-09-16T11:43:44.293 INFO:teuthology.orchestra.run.smithi070.stdout:1158 write 0x8d5e983 thru 0x8d68d78 (0xa3f6 bytes) 2024-09-16T11:43:44.293 INFO:teuthology.orchestra.run.smithi070.stdout:1159 writesame 0xc5947dd thru 0xc5a0a9f data_size 0x1(0xc2c3 bytes) 2024-09-16T11:43:44.293 INFO:teuthology.orchestra.run.smithi070.stdout:1160 write 0xd816b09 thru 0xd818efd (0x23f5 bytes) 2024-09-16T11:43:44.293 INFO:teuthology.orchestra.run.smithi070.stdout:1161 write 0xbe71d05 thru 0xbe7f964 (0xdc60 bytes) 2024-09-16T11:43:44.293 INFO:teuthology.orchestra.run.smithi070.stdout:1163 read 0x9206248 thru 0x9209481 (0x323a bytes) 2024-09-16T11:43:44.293 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.293 INFO:teuthology.orchestra.run.smithi070.stdout:1165 writesame 0x8635120 thru 0x86360c9 data_size 0x5(0xfaa bytes) 2024-09-16T11:43:44.293 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.293 INFO:teuthology.orchestra.run.smithi070.stdout:1171 read 0x3a3ac97 thru 0x3a4a658 (0xf9c2 bytes) 2024-09-16T11:43:44.293 INFO:teuthology.orchestra.run.smithi070.stdout:1172 punch from 0x5302d11 to 0x5309d22, (0x7011 bytes) 2024-09-16T11:43:44.294 INFO:teuthology.orchestra.run.smithi070.stdout:1175 punch from 0x66c0836 to 0x66caca5, (0xa46f bytes) 2024-09-16T11:43:44.294 INFO:teuthology.orchestra.run.smithi070.stdout:1176 writesame 0x500f6c3 thru 0x501c6f7 data_size 0x6d(0xd035 bytes) 2024-09-16T11:43:44.294 INFO:teuthology.orchestra.run.smithi070.stdout:1177 trunc from 0xd818efe to 0x2e00017 2024-09-16T11:43:44.295 INFO:teuthology.orchestra.run.smithi070.stdout:1178 read 0x10d1f40 thru 0x10d9ffb (0x80bc bytes) 2024-09-16T11:43:44.295 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:44.290+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:44.296 INFO:teuthology.orchestra.run.smithi070.stdout:1179 read 0x1c2c9f7 thru 0x1c37a8e (0xb098 bytes) 2024-09-16T11:43:44.296 INFO:teuthology.orchestra.run.smithi070.stdout:1180 write 0x1baafe8 thru 0x1bb1fdc (0x6ff5 bytes) 2024-09-16T11:43:44.296 INFO:teuthology.orchestra.run.smithi070.stdout:1181 read 0x156d167 thru 0x1571388 (0x4222 bytes) 2024-09-16T11:43:44.296 INFO:teuthology.orchestra.run.smithi070.stdout:1183 read 0x27af0fe thru 0x27bc2cc (0xd1cf bytes) 2024-09-16T11:43:44.296 INFO:teuthology.orchestra.run.smithi070.stdout:1184 read 0x23b1a2b thru 0x23c1176 (0xf74c bytes) 2024-09-16T11:43:44.296 INFO:teuthology.orchestra.run.smithi070.stdout:1185 read 0x143a68d thru 0x143dae2 (0x3456 bytes) 2024-09-16T11:43:44.297 INFO:teuthology.orchestra.run.smithi070.stdout:1186 trunc from 0x2e00017 to 0x3fdbd1c 2024-09-16T11:43:44.302 INFO:teuthology.orchestra.run.smithi070.stdout:1187 trunc from 0x3fdbd1c to 0x331ddc6 2024-09-16T11:43:44.321 INFO:teuthology.orchestra.run.smithi070.stdout:1189 writesame 0x82a286c thru 0x82b12a1 data_size 0x9(0xea36 bytes) 2024-09-16T11:43:44.321 INFO:teuthology.orchestra.run.smithi070.stdout:1190 write 0x66b243d thru 0x66b78a7 (0x546b bytes) 2024-09-16T11:43:44.332 INFO:teuthology.orchestra.run.smithi070.stdout:1191 writesame 0xaf12194 thru 0xaf21da7 data_size 0x49(0xfc14 bytes) 2024-09-16T11:43:44.339 INFO:teuthology.orchestra.run.smithi070.stdout:1193 write 0xc5626fc thru 0xc56c7d2 (0xa0d7 bytes) 2024-09-16T11:43:44.348 INFO:teuthology.orchestra.run.smithi070.stdout:1194 trunc from 0xc56c7d3 to 0x671795b 2024-09-16T11:43:44.358 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.367 INFO:teuthology.orchestra.run.smithi070.stdout:1197 writesame 0xa475984 thru 0xa475a72 data_size 0x1(0xef bytes) 2024-09-16T11:43:44.367 INFO:teuthology.orchestra.run.smithi070.stdout:1198 read 0x5d03fea thru 0x5d0bead (0x7ec4 bytes) 2024-09-16T11:43:44.367 INFO:teuthology.orchestra.run.smithi070.stdout:1201 read 0x9f08a9a thru 0x9f0b936 (0x2e9d bytes) 2024-09-16T11:43:44.368 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.368 INFO:teuthology.orchestra.run.smithi070.stdout:1206 write 0x4077d60 thru 0x4084c7b (0xcf1c bytes) 2024-09-16T11:43:44.369 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.375 INFO:teuthology.orchestra.run.smithi070.stdout:1209 writesame 0xb32fbf9 thru 0xb33f77e data_size 0x89(0xfb86 bytes) 2024-09-16T11:43:44.382 INFO:teuthology.orchestra.run.smithi070.stdout:1210 writesame 0xd577a01 thru 0xd5816d4 data_size 0x2(0x9cd4 bytes) 2024-09-16T11:43:44.382 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.382 INFO:teuthology.orchestra.run.smithi070.stdout:1213 write 0x537ff00 thru 0x53858f6 (0x59f7 bytes) 2024-09-16T11:43:44.384 INFO:teuthology.orchestra.run.smithi070.stdout:1214 read 0x56df3b6 thru 0x56dfdfc (0xa47 bytes) 2024-09-16T11:43:44.385 INFO:teuthology.orchestra.run.smithi070.stdout:1215 write 0x358a46e thru 0x3593f84 (0x9b17 bytes) 2024-09-16T11:43:44.393 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:44.391+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:44.402 INFO:teuthology.orchestra.run.smithi070.stdout:1216 trunc from 0xd5816d5 to 0x6c596f 2024-09-16T11:43:44.406 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:44.406 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:44.414 INFO:teuthology.orchestra.run.smithi070.stdout:1217 read 0x4d69cb thru 0x4d6b2d (0x163 bytes) 2024-09-16T11:43:44.415 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:43:44.436 INFO:teuthology.orchestra.run.smithi070.stdout:1219 write 0xd92d96b thru 0xd93bb1a (0xe1b0 bytes) 2024-09-16T11:43:44.438 INFO:teuthology.orchestra.run.smithi070.stdout:1220 punch from 0xa7247a7 to 0xa72e91c, (0xa175 bytes) 2024-09-16T11:43:44.439 INFO:teuthology.orchestra.run.smithi070.stdout:1221 read 0xa23acf3 thru 0xa2483c9 (0xd6d7 bytes) 2024-09-16T11:43:44.439 INFO:teuthology.orchestra.run.smithi070.stdout:1222 write 0x55416e4 thru 0x5543abf (0x23dc bytes) 2024-09-16T11:43:44.441 INFO:teuthology.orchestra.run.smithi070.stdout:1224 write 0x3f4da7d thru 0x3f4f2ed (0x1871 bytes) 2024-09-16T11:43:44.443 INFO:teuthology.orchestra.run.smithi070.stdout:1225 read 0xb38edff thru 0xb39b638 (0xc83a bytes) 2024-09-16T11:43:44.443 INFO:teuthology.orchestra.run.smithi070.stdout:1227 punch from 0x3f06141 to 0x3f07d5d, (0x1c1c bytes) 2024-09-16T11:43:44.443 INFO:teuthology.orchestra.run.smithi070.stdout:1228 write 0x9bbea6d thru 0x9bc0425 (0x19b9 bytes) 2024-09-16T11:43:44.446 INFO:teuthology.orchestra.run.smithi070.stdout:1229 write 0x5cb8b7f thru 0x5cc7922 (0xeda4 bytes) 2024-09-16T11:43:44.450 INFO:teuthology.orchestra.run.smithi070.stdout:1232 punch from 0x501b458 to 0x502955d, (0xe105 bytes) 2024-09-16T11:43:44.450 INFO:teuthology.orchestra.run.smithi070.stdout:1234 write 0x38d4f00 thru 0x38e388b (0xe98c bytes) 2024-09-16T11:43:44.452 INFO:teuthology.orchestra.run.smithi070.stdout:1235 write 0x118bd54 thru 0x118cb30 (0xddd bytes) 2024-09-16T11:43:44.453 INFO:teuthology.orchestra.run.smithi070.stdout:1236 writesame 0x967931 thru 0x968d75 data_size 0x1(0x1445 bytes) 2024-09-16T11:43:44.455 INFO:teuthology.orchestra.run.smithi070.stdout:1237 write 0x4103adc thru 0x410edb5 (0xb2da bytes) 2024-09-16T11:43:44.457 INFO:teuthology.orchestra.run.smithi070.stdout:1238 read 0x99f6843 thru 0x99f6ecd (0x68b bytes) 2024-09-16T11:43:44.467 INFO:teuthology.orchestra.run.smithi070.stdout:1239 trunc from 0xd93bb1b to 0x577f60c 2024-09-16T11:43:44.477 INFO:teuthology.orchestra.run.smithi070.stdout:1240 write 0x47a13df thru 0x47a4bdc (0x37fe bytes) 2024-09-16T11:43:44.478 INFO:teuthology.orchestra.run.smithi070.stdout:1242 read 0x3a9398b thru 0x3a9657b (0x2bf1 bytes) 2024-09-16T11:43:44.478 INFO:teuthology.orchestra.run.smithi070.stdout:1243 read 0x4522110 thru 0x452418e (0x207f bytes) 2024-09-16T11:43:44.486 INFO:teuthology.orchestra.run.smithi070.stdout:1244 write 0x7c5660a thru 0x7c5921d (0x2c14 bytes) 2024-09-16T11:43:44.488 INFO:teuthology.orchestra.run.smithi070.stdout:1246 read 0x6f3b5b9 thru 0x6f4196b (0x63b3 bytes) 2024-09-16T11:43:44.488 INFO:teuthology.orchestra.run.smithi070.stdout:1248 punch from 0x2018c6f to 0x201ac05, (0x1f96 bytes) 2024-09-16T11:43:44.488 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.488 INFO:teuthology.orchestra.run.smithi070.stdout:1251 writesame 0xf8eb02 thru 0xf9df94 data_size 0x1d(0xf493 bytes) 2024-09-16T11:43:44.494 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:44.492+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:44.498 INFO:teuthology.orchestra.run.smithi070.stdout:1252 write 0xaa6e00d thru 0xaa7c17b (0xe16f bytes) 2024-09-16T11:43:44.500 INFO:teuthology.orchestra.run.smithi070.stdout:1253 write 0x102a533 thru 0x1037fc8 (0xda96 bytes) 2024-09-16T11:43:44.508 INFO:teuthology.orchestra.run.smithi070.stdout:1254 write 0xb64e53b thru 0xb64ed63 (0x829 bytes) 2024-09-16T11:43:44.510 INFO:teuthology.orchestra.run.smithi070.stdout:1255 write 0x380b90 thru 0x38fa14 (0xee85 bytes) 2024-09-16T11:43:44.516 INFO:teuthology.orchestra.run.smithi070.stdout:1256 trunc from 0xb64ed64 to 0xed73508 2024-09-16T11:43:44.521 INFO:teuthology.orchestra.run.smithi070.stdout:1257 writesame 0x13d8770 thru 0x13e1a60 data_size 0x1(0x92f1 bytes) 2024-09-16T11:43:44.534 INFO:teuthology.orchestra.run.smithi070.stdout:1258 write 0x9e29eb8 thru 0x9e2d6cb (0x3814 bytes) 2024-09-16T11:43:44.537 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.537 INFO:teuthology.orchestra.run.smithi070.stdout:1263 writesame 0x3990cbe thru 0x39a0a3f data_size 0x25(0xfd82 bytes) 2024-09-16T11:43:44.539 INFO:teuthology.orchestra.run.smithi070.stdout:1264 read 0x1e42cb9 thru 0x1e4858a (0x58d2 bytes) 2024-09-16T11:43:44.546 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:44.546 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:44.551 INFO:teuthology.orchestra.run.smithi070.stdout:1265 trunc from 0xed73508 to 0x5335f17 2024-09-16T11:43:44.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:43:44.561 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.577 INFO:teuthology.orchestra.run.smithi070.stdout:1268 write 0xe208875 thru 0xe211b3f (0x92cb bytes) 2024-09-16T11:43:44.579 INFO:teuthology.orchestra.run.smithi070.stdout:1269 read 0x9d2601 thru 0x9e0277 (0xdc77 bytes) 2024-09-16T11:43:44.580 INFO:teuthology.orchestra.run.smithi070.stdout:1270 read 0x7fbfc12 thru 0x7fc7844 (0x7c33 bytes) 2024-09-16T11:43:44.580 INFO:teuthology.orchestra.run.smithi070.stdout:1272 writesame 0x861a23f thru 0x8624201 data_size 0x1(0x9fc3 bytes) 2024-09-16T11:43:44.595 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:44.593+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:44.601 INFO:teuthology.orchestra.run.smithi070.stdout:1274 trunc from 0xe211b40 to 0xd07f1b0 2024-09-16T11:43:44.610 INFO:teuthology.orchestra.run.smithi070.stdout:1275 read 0x1cd794e thru 0x1cded52 (0x7405 bytes) 2024-09-16T11:43:44.610 INFO:teuthology.orchestra.run.smithi070.stdout:1276 write 0x4497161 thru 0x44a3eea (0xcd8a bytes) 2024-09-16T11:43:44.617 INFO:teuthology.orchestra.run.smithi070.stdout:1277 trunc from 0xd07f1b0 to 0x8a62a42 2024-09-16T11:43:44.630 INFO:teuthology.orchestra.run.smithi070.stdout:1278 writesame 0x9ca7174 thru 0x9caf53e data_size 0x1(0x83cb bytes) 2024-09-16T11:43:44.630 INFO:teuthology.orchestra.run.smithi070.stdout:1279 writesame 0x5273455 thru 0x52822cb data_size 0x85(0xee77 bytes) 2024-09-16T11:43:44.631 INFO:teuthology.orchestra.run.smithi070.stdout:1280 write 0x71d5330 thru 0x71dd959 (0x862a bytes) 2024-09-16T11:43:44.633 INFO:teuthology.orchestra.run.smithi070.stdout:1281 read 0x44725a4 thru 0x44802f4 (0xdd51 bytes) 2024-09-16T11:43:44.633 INFO:teuthology.orchestra.run.smithi070.stdout:1282 writesame 0x1168b51 thru 0x1170e78 data_size 0xc(0x8328 bytes) 2024-09-16T11:43:44.635 INFO:teuthology.orchestra.run.smithi070.stdout:1283 read 0x30bedc9 thru 0x30cc680 (0xd8b8 bytes) 2024-09-16T11:43:44.645 INFO:teuthology.orchestra.run.smithi070.stdout:1285 trunc from 0x9caf53f to 0x2405a4e 2024-09-16T11:43:44.656 INFO:teuthology.orchestra.run.smithi070.stdout:1286 punch from 0x59255a to 0x5a13f8, (0xee9e bytes) 2024-09-16T11:43:44.670 INFO:teuthology.orchestra.run.smithi070.stdout:1288 write 0x9908acd thru 0x991477d (0xbcb1 bytes) 2024-09-16T11:43:44.673 INFO:teuthology.orchestra.run.smithi070.stdout:1290 read 0x721ce21 thru 0x7227ee0 (0xb0c0 bytes) 2024-09-16T11:43:44.673 INFO:teuthology.orchestra.run.smithi070.stdout:1293 punch from 0x22256b9 to 0x2232930, (0xd277 bytes) 2024-09-16T11:43:44.673 INFO:teuthology.orchestra.run.smithi070.stdout:1294 write 0x471a1f4 thru 0x4726476 (0xc283 bytes) 2024-09-16T11:43:44.674 INFO:teuthology.orchestra.run.smithi070.stdout:1295 read 0x1872d29 thru 0x18819de (0xecb6 bytes) 2024-09-16T11:43:44.675 INFO:teuthology.orchestra.run.smithi070.stdout:1296 writesame 0x6429933 thru 0x6436783 data_size 0x1(0xce51 bytes) 2024-09-16T11:43:44.676 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:44.676 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:44.686 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:43:44.694 INFO:teuthology.orchestra.run.smithi070.stdout:1297 read 0x18caef8 thru 0x18cf43f (0x4548 bytes) 2024-09-16T11:43:44.695 INFO:teuthology.orchestra.run.smithi070.stdout:1298 write 0x15b86e2 thru 0x15c50ec (0xca0b bytes) 2024-09-16T11:43:44.696 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:44.694+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:44.704 INFO:teuthology.orchestra.run.smithi070.stdout:1300 trunc from 0x991477e to 0x36c6ba9 2024-09-16T11:43:44.722 INFO:teuthology.orchestra.run.smithi070.stdout:1301 write 0x5a24ded thru 0x5a343da (0xf5ee bytes) 2024-09-16T11:43:44.724 INFO:teuthology.orchestra.run.smithi070.stdout:1302 read 0x301a932 thru 0x3025959 (0xb028 bytes) 2024-09-16T11:43:44.740 INFO:teuthology.orchestra.run.smithi070.stdout:1303 write 0xe6d2584 thru 0xe6e0896 (0xe313 bytes) 2024-09-16T11:43:44.744 INFO:teuthology.orchestra.run.smithi070.stdout:1304 read 0x844ced6 thru 0x8455a4c (0x8b77 bytes) 2024-09-16T11:43:44.745 INFO:teuthology.orchestra.run.smithi070.stdout:1305 punch from 0xb839a32 to 0xb83e87a, (0x4e48 bytes) 2024-09-16T11:43:44.745 INFO:teuthology.orchestra.run.smithi070.stdout:1306 read 0xb3ff289 thru 0xb400815 (0x158d bytes) 2024-09-16T11:43:44.745 INFO:teuthology.orchestra.run.smithi070.stdout:1307 read 0xe4e2512 thru 0xe4e3b5f (0x164e bytes) 2024-09-16T11:43:44.745 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.745 INFO:teuthology.orchestra.run.smithi070.stdout:1310 writesame 0x543b412 thru 0x5445185 data_size 0x6(0x9d74 bytes) 2024-09-16T11:43:44.749 INFO:teuthology.orchestra.run.smithi070.stdout:1312 read 0x1ab053f thru 0x1abc626 (0xc0e8 bytes) 2024-09-16T11:43:44.750 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.750 INFO:teuthology.orchestra.run.smithi070.stdout:1314 writesame 0xe7e146 thru 0xe8788d data_size 0x8(0x9748 bytes) 2024-09-16T11:43:44.753 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.753 INFO:teuthology.orchestra.run.smithi070.stdout:1318 punch from 0xb2fc756 to 0xb30ae8c, (0xe736 bytes) 2024-09-16T11:43:44.753 INFO:teuthology.orchestra.run.smithi070.stdout:1319 read 0x2626d48 thru 0x2635bba (0xee73 bytes) 2024-09-16T11:43:44.753 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.754 INFO:teuthology.orchestra.run.smithi070.stdout:1321 write 0xaaf9e11 thru 0xab02bbe (0x8dae bytes) 2024-09-16T11:43:44.757 INFO:teuthology.orchestra.run.smithi070.stdout:1322 punch from 0x6b350e8 to 0x6b3f70a, (0xa622 bytes) 2024-09-16T11:43:44.762 INFO:teuthology.orchestra.run.smithi070.stdout:1323 write 0xec5e09d thru 0xec60a7d (0x29e1 bytes) 2024-09-16T11:43:44.765 INFO:teuthology.orchestra.run.smithi070.stdout:1326 writesame 0xadffd8b thru 0xae00a64 data_size 0xe(0xcda bytes) 2024-09-16T11:43:44.768 INFO:teuthology.orchestra.run.smithi070.stdout:1327 read 0x7fca1d7 thru 0x7fcb70e (0x1538 bytes) 2024-09-16T11:43:44.768 INFO:teuthology.orchestra.run.smithi070.stdout:1328 read 0x698ba67 thru 0x6997872 (0xbe0c bytes) 2024-09-16T11:43:44.768 INFO:teuthology.orchestra.run.smithi070.stdout:1329 writesame 0x9aee49d thru 0x9af1e36 data_size 0x2(0x399a bytes) 2024-09-16T11:43:44.774 INFO:teuthology.orchestra.run.smithi070.stdout:1330 read 0x2a97b2b thru 0x2aa772b (0xfc01 bytes) 2024-09-16T11:43:44.775 INFO:teuthology.orchestra.run.smithi070.stdout:1331 flatten 2024-09-16T11:43:44.780 INFO:teuthology.orchestra.run.smithi070.stdout:1335 writesame 0x50bede2 thru 0x50c1fa5 data_size 0x46(0x31c4 bytes) 2024-09-16T11:43:44.782 INFO:teuthology.orchestra.run.smithi070.stdout:1337 punch from 0x76de3c0 to 0x76e2898, (0x44d8 bytes) 2024-09-16T11:43:44.797 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:44.794+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:44.798 INFO:teuthology.orchestra.run.smithi070.stdout:1338 trunc from 0xec60a7e to 0x1e01a1d 2024-09-16T11:43:44.811 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:44.811 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:44.821 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:43:44.837 INFO:teuthology.orchestra.run.smithi070.stdout:1339 write 0xe1485c8 thru 0xe153297 (0xacd0 bytes) 2024-09-16T11:43:44.839 INFO:teuthology.orchestra.run.smithi070.stdout:1340 read 0x250c92 thru 0x250cfd (0x6c bytes) 2024-09-16T11:43:44.840 INFO:teuthology.orchestra.run.smithi070.stdout:1341 read 0xd34f94f thru 0xd3551e3 (0x5895 bytes) 2024-09-16T11:43:44.840 INFO:teuthology.orchestra.run.smithi070.stdout:1343 punch from 0x62ada3e to 0x62b8cbe, (0xb280 bytes) 2024-09-16T11:43:44.842 INFO:teuthology.orchestra.run.smithi070.stdout:1345 trunc from 0xe153298 to 0xc92fb22 2024-09-16T11:43:44.851 INFO:teuthology.orchestra.run.smithi070.stdout:1346 write 0x16cabd6 thru 0x16d6192 (0xb5bd bytes) 2024-09-16T11:43:44.852 INFO:teuthology.orchestra.run.smithi070.stdout:1347 read 0x3fc0602 thru 0x3fcfd96 (0xf795 bytes) 2024-09-16T11:43:44.859 INFO:teuthology.orchestra.run.smithi070.stdout:1348 trunc from 0xc92fb22 to 0x77b0736 2024-09-16T11:43:44.876 INFO:teuthology.orchestra.run.smithi070.stdout:1350 write 0xdd7acad thru 0xdd857c8 (0xab1c bytes) 2024-09-16T11:43:44.880 INFO:teuthology.orchestra.run.smithi070.stdout:1351 write 0x52c2f6f thru 0x52c65dc (0x366e bytes) 2024-09-16T11:43:44.882 INFO:teuthology.orchestra.run.smithi070.stdout:1352 writesame 0x2d8088a thru 0x2d8fe65 data_size 0x1e(0xf5dc bytes) 2024-09-16T11:43:44.884 INFO:teuthology.orchestra.run.smithi070.stdout:1353 read 0x7b07a68 thru 0x7b0a71f (0x2cb8 bytes) 2024-09-16T11:43:44.892 INFO:teuthology.orchestra.run.smithi070.stdout:1355 trunc from 0xdd857c9 to 0x70e7072 2024-09-16T11:43:44.897 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:44.896+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:44.908 INFO:teuthology.orchestra.run.smithi070.stdout:1356 write 0x823bc11 thru 0x8243f31 (0x8321 bytes) 2024-09-16T11:43:44.920 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:43:44.921 INFO:teuthology.orchestra.run.smithi070.stdout:1357 trunc from 0x8243f32 to 0x120e4cc 2024-09-16T11:43:44.935 INFO:teuthology.orchestra.run.smithi070.stdout:1358 write 0xdb0018 thru 0xdb9a85 (0x9a6e bytes) 2024-09-16T11:43:44.936 INFO:teuthology.orchestra.run.smithi070.stdout:1360 punch from 0xee4357 to 0xee8706, (0x43af bytes) 2024-09-16T11:43:44.950 INFO:teuthology.orchestra.run.smithi070.stdout:1361 writesame 0x6ee56df thru 0x6ef06df data_size 0x17(0xb001 bytes) 2024-09-16T11:43:44.950 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:44.950 INFO:teuthology.orchestra.run.smithi070.stdout:1363 punch from 0x45981 to 0x4c695, (0x6d14 bytes) 2024-09-16T11:43:44.950 INFO:teuthology.orchestra.run.smithi070.stdout:1364 punch from 0x2e36f67 to 0x2e44f66, (0xdfff bytes) 2024-09-16T11:43:44.961 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:44.961 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:44.965 INFO:teuthology.orchestra.run.smithi070.stdout:1365 write 0xe7d7995 thru 0xe7e39f3 (0xc05f bytes) 2024-09-16T11:43:44.969 INFO:teuthology.orchestra.run.smithi070.stdout:1366 writesame 0x82194f8 thru 0x821ef17 data_size 0x38(0x5a20 bytes) 2024-09-16T11:43:44.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:43:44.987 INFO:teuthology.orchestra.run.smithi070.stdout:1367 trunc from 0xe7e39f4 to 0x1bbfd06 2024-09-16T11:43:45.000 INFO:teuthology.orchestra.run.smithi070.stdout:1368 punch from 0x7fb9b9 to 0x804a39, (0x9080 bytes) 2024-09-16T11:43:45.001 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:44.998+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:45.015 INFO:teuthology.orchestra.run.smithi070.stdout:1370 writesame 0x9a347c5 thru 0x9a34979 data_size 0x1(0x1b5 bytes) 2024-09-16T11:43:45.015 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.015 INFO:teuthology.orchestra.run.smithi070.stdout:1372 read 0x7543cd7 thru 0x754a6be (0x69e8 bytes) 2024-09-16T11:43:45.015 INFO:teuthology.orchestra.run.smithi070.stdout:1373 read 0x3507aea thru 0x3514e59 (0xd370 bytes) 2024-09-16T11:43:45.026 INFO:teuthology.orchestra.run.smithi070.stdout:1374 writesame 0xda96c50 thru 0xda9ef4a data_size 0x1(0x82fb bytes) 2024-09-16T11:43:45.026 INFO:teuthology.orchestra.run.smithi070.stdout:1376 read 0xb085468 thru 0xb08a7c8 (0x5361 bytes) 2024-09-16T11:43:45.026 INFO:teuthology.orchestra.run.smithi070.stdout:1377 read 0x1b4bf9d thru 0x1b54414 (0x8478 bytes) 2024-09-16T11:43:45.026 INFO:teuthology.orchestra.run.smithi070.stdout:1378 write 0xd2d3fc2 thru 0xd2dcb00 (0x8b3f bytes) 2024-09-16T11:43:45.029 INFO:teuthology.orchestra.run.smithi070.stdout:1380 write 0xc8f7fe9 thru 0xc8fcbc5 (0x4bdd bytes) 2024-09-16T11:43:45.032 INFO:teuthology.orchestra.run.smithi070.stdout:1381 read 0x39c4cc0 thru 0x39cade4 (0x6125 bytes) 2024-09-16T11:43:45.032 INFO:teuthology.orchestra.run.smithi070.stdout:1384 read 0x42e2d91 thru 0x42ef5a6 (0xc816 bytes) 2024-09-16T11:43:45.032 INFO:teuthology.orchestra.run.smithi070.stdout:1386 write 0x19cffdd thru 0x19dd4b9 (0xd4dd bytes) 2024-09-16T11:43:45.034 INFO:teuthology.orchestra.run.smithi070.stdout:1387 write 0x3a51af3 thru 0x3a599b7 (0x7ec5 bytes) 2024-09-16T11:43:45.036 INFO:teuthology.orchestra.run.smithi070.stdout:1388 punch from 0x85f302f to 0x85f3cbc, (0xc8d bytes) 2024-09-16T11:43:45.036 INFO:teuthology.orchestra.run.smithi070.stdout:1389 writesame 0xbf89014 thru 0xbf94263 data_size 0x48(0xb250 bytes) 2024-09-16T11:43:45.039 INFO:teuthology.orchestra.run.smithi070.stdout:1391 write 0xa2fc32a thru 0xa3066f4 (0xa3cb bytes) 2024-09-16T11:43:45.044 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.044 INFO:teuthology.orchestra.run.smithi070.stdout:1393 read 0x820c11b thru 0x8218c0c (0xcaf2 bytes) 2024-09-16T11:43:45.044 INFO:teuthology.orchestra.run.smithi070.stdout:1394 writesame 0xc0c24f5 thru 0xc0c2ca0 data_size 0x2(0x7ac bytes) 2024-09-16T11:43:45.045 INFO:teuthology.orchestra.run.smithi070.stdout:1395 write 0x714028c thru 0x714f47a (0xf1ef bytes) 2024-09-16T11:43:45.049 INFO:teuthology.orchestra.run.smithi070.stdout:1398 read 0x2e18d08 thru 0x2e1d825 (0x4b1e bytes) 2024-09-16T11:43:45.049 INFO:teuthology.orchestra.run.smithi070.stdout:1399 write 0x7249fde thru 0x725307a (0x909d bytes) 2024-09-16T11:43:45.057 INFO:teuthology.orchestra.run.smithi070.stdout:1400 trunc from 0xda9ef4b to 0xa02903e 2024-09-16T11:43:45.064 INFO:teuthology.orchestra.run.smithi070.stdout:1401 writesame 0x3d7e44b thru 0x3d86946 data_size 0x6(0x84fc bytes) 2024-09-16T11:43:45.080 INFO:teuthology.orchestra.run.smithi070.stdout:1404 trunc from 0xa02903e to 0x1371eaf 2024-09-16T11:43:45.097 INFO:teuthology.orchestra.run.smithi070.stdout:1405 trunc from 0x1371eaf to 0x4b219a7 2024-09-16T11:43:45.099 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:45.098+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:45.116 INFO:teuthology.orchestra.run.smithi070.stdout:1406 write 0xa91a13e thru 0xa91b97f (0x1842 bytes) 2024-09-16T11:43:45.118 INFO:teuthology.orchestra.run.smithi070.stdout:1407 read 0x950e011 thru 0x95112f2 (0x32e2 bytes) 2024-09-16T11:43:45.123 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:45.123 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:45.123 INFO:teuthology.orchestra.run.smithi070.stdout:1408 write 0xacd501b thru 0xacd858a (0x3570 bytes) 2024-09-16T11:43:45.126 INFO:teuthology.orchestra.run.smithi070.stdout:1409 write 0x30c7e78 thru 0x30d56bf (0xd848 bytes) 2024-09-16T11:43:45.127 INFO:teuthology.orchestra.run.smithi070.stdout:1410 read 0xac253a4 thru 0xac2dff9 (0x8c56 bytes) 2024-09-16T11:43:45.130 INFO:teuthology.orchestra.run.smithi070.stdout:1411 trunc from 0xacd858b to 0xcba30f7 2024-09-16T11:43:45.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:43:45.136 INFO:teuthology.orchestra.run.smithi070.stdout:1414 write 0x50801ad thru 0x50866f2 (0x6546 bytes) 2024-09-16T11:43:45.148 INFO:teuthology.orchestra.run.smithi070.stdout:1415 trunc from 0xcba30f7 to 0x537246c 2024-09-16T11:43:45.158 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.158 INFO:teuthology.orchestra.run.smithi070.stdout:1417 read 0x2e9fd19 thru 0x2ea4e48 (0x5130 bytes) 2024-09-16T11:43:45.168 INFO:teuthology.orchestra.run.smithi070.stdout:1418 trunc from 0x537246c to 0xcf95fc3 2024-09-16T11:43:45.173 INFO:teuthology.orchestra.run.smithi070.stdout:1420 writesame 0x47ca3bd thru 0x47cff37 data_size 0x1(0x5b7b bytes) 2024-09-16T11:43:45.184 INFO:teuthology.orchestra.run.smithi070.stdout:1421 punch from 0xaf4ce8c to 0xaf5186d, (0x49e1 bytes) 2024-09-16T11:43:45.185 INFO:teuthology.orchestra.run.smithi070.stdout:1422 trunc from 0xcf95fc3 to 0xc82acef 2024-09-16T11:43:45.191 INFO:teuthology.orchestra.run.smithi070.stdout:1426 writesame 0x798c612 thru 0x799714d data_size 0x4e(0xab3c bytes) 2024-09-16T11:43:45.194 INFO:teuthology.orchestra.run.smithi070.stdout:1428 read 0x17be0c7 thru 0x17c08f3 (0x282d bytes) 2024-09-16T11:43:45.194 INFO:teuthology.orchestra.run.smithi070.stdout:1429 read 0x1c6e926 thru 0x1c7b8b1 (0xcf8c bytes) 2024-09-16T11:43:45.194 INFO:teuthology.orchestra.run.smithi070.stdout:1430 writesame 0x33c16a6 thru 0x33c8250 data_size 0x1(0x6bab bytes) 2024-09-16T11:43:45.199 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:45.198+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:45.218 INFO:teuthology.orchestra.run.smithi070.stdout:1431 trunc from 0xc82acef to 0x339f1d6 2024-09-16T11:43:45.230 INFO:teuthology.orchestra.run.smithi070.stdout:1432 punch from 0x1af65e6 to 0x1af859d, (0x1fb7 bytes) 2024-09-16T11:43:45.230 INFO:teuthology.orchestra.run.smithi070.stdout:1435 writesame 0xedc50f thru 0xeeb955 data_size 0x37(0xf447 bytes) 2024-09-16T11:43:45.244 INFO:teuthology.orchestra.run.smithi070.stdout:1436 writesame 0x8644567 thru 0x86517bd data_size 0x1f(0xd257 bytes) 2024-09-16T11:43:45.244 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.244 INFO:teuthology.orchestra.run.smithi070.stdout:1439 writesame 0x8c2122 thru 0x8cc9d4 data_size 0x1(0xa8b3 bytes) 2024-09-16T11:43:45.251 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:45.251 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:45.259 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.260 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:43:45.267 INFO:teuthology.orchestra.run.smithi070.stdout:1444 write 0xa3502ac thru 0xa3560c5 (0x5e1a bytes) 2024-09-16T11:43:45.270 INFO:teuthology.orchestra.run.smithi070.stdout:1445 writesame 0x912dcec thru 0x913c8ce data_size 0x1(0xebe3 bytes) 2024-09-16T11:43:45.298 INFO:teuthology.orchestra.run.smithi070.stdout:1446 read 0x1daf80 thru 0x1de74b (0x37cc bytes) 2024-09-16T11:43:45.298 INFO:teuthology.orchestra.run.smithi070.stdout:1449 writesame 0x1bc0881 thru 0x1bc2b91 data_size 0x1(0x2311 bytes) 2024-09-16T11:43:45.300 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:45.299+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:45.303 INFO:teuthology.orchestra.run.smithi070.stdout:1450 write 0x9e37e2b thru 0x9e3de64 (0x603a bytes) 2024-09-16T11:43:45.306 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.306 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.306 INFO:teuthology.orchestra.run.smithi070.stdout:1454 writesame 0x598f8ca thru 0x59924d8 data_size 0x1(0x2c0f bytes) 2024-09-16T11:43:45.311 INFO:teuthology.orchestra.run.smithi070.stdout:1455 punch from 0x9bcfb77 to 0x9bd3249, (0x36d2 bytes) 2024-09-16T11:43:45.311 INFO:teuthology.orchestra.run.smithi070.stdout:1456 write 0x33a0bef thru 0x33afd4e (0xf160 bytes) 2024-09-16T11:43:45.313 INFO:teuthology.orchestra.run.smithi070.stdout:1457 write 0x53f6507 thru 0x53fd0f9 (0x6bf3 bytes) 2024-09-16T11:43:45.314 INFO:teuthology.orchestra.run.smithi070.stdout:1458 read 0x381c90 thru 0x38f0b2 (0xd423 bytes) 2024-09-16T11:43:45.324 INFO:teuthology.orchestra.run.smithi070.stdout:1460 write 0xb593bde thru 0xb5978e4 (0x3d07 bytes) 2024-09-16T11:43:45.326 INFO:teuthology.orchestra.run.smithi070.stdout:1462 trunc from 0xb5978e5 to 0xa262fea 2024-09-16T11:43:45.332 INFO:teuthology.orchestra.run.smithi070.stdout:1463 write 0x998fcbf thru 0x999739e (0x76e0 bytes) 2024-09-16T11:43:45.334 INFO:teuthology.orchestra.run.smithi070.stdout:1464 write 0x6b21869 thru 0x6b29e16 (0x85ae bytes) 2024-09-16T11:43:45.337 INFO:teuthology.orchestra.run.smithi070.stdout:1465 write 0x7ea0137 thru 0x7eadae1 (0xd9ab bytes) 2024-09-16T11:43:45.339 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.339 INFO:teuthology.orchestra.run.smithi070.stdout:1468 punch from 0x4d05f32 to 0x4d0fc0c, (0x9cda bytes) 2024-09-16T11:43:45.350 INFO:teuthology.orchestra.run.smithi070.stdout:1469 trunc from 0xa262fea to 0x2d4ee5 2024-09-16T11:43:45.365 INFO:teuthology.orchestra.run.smithi070.stdout:1470 punch from 0x20a75e to 0x2185fb, (0xde9d bytes) 2024-09-16T11:43:45.365 INFO:teuthology.orchestra.run.smithi070.stdout:1472 punch from 0x19983f to 0x1a6f96, (0xd757 bytes) 2024-09-16T11:43:45.375 INFO:teuthology.orchestra.run.smithi070.stdout:1473 writesame 0x4b52b4b thru 0x4b52efc data_size 0x2(0x3b2 bytes) 2024-09-16T11:43:45.377 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:45.378 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:45.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:43:45.391 INFO:teuthology.orchestra.run.smithi070.stdout:1474 write 0xcd865a8 thru 0xcd961a4 (0xfbfd bytes) 2024-09-16T11:43:45.397 INFO:teuthology.orchestra.run.smithi070.stdout:1475 writesame 0x36f837a thru 0x3707fc0 data_size 0x1d(0xfc47 bytes) 2024-09-16T11:43:45.399 INFO:teuthology.orchestra.run.smithi070.stdout:1476 write 0xb37369c thru 0xb37e96c (0xb2d1 bytes) 2024-09-16T11:43:45.401 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:45.400+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:45.402 INFO:teuthology.orchestra.run.smithi070.stdout:1480 read 0xaedca22 thru 0xaee1cc0 (0x529f bytes) 2024-09-16T11:43:45.402 INFO:teuthology.orchestra.run.smithi070.stdout:1481 writesame 0xb31b0d9 thru 0xb329d87 data_size 0x13(0xecaf bytes) 2024-09-16T11:43:45.408 INFO:teuthology.orchestra.run.smithi070.stdout:1482 write 0xbcd3d7c thru 0xbcd497e (0xc03 bytes) 2024-09-16T11:43:45.418 INFO:teuthology.orchestra.run.smithi070.stdout:1483 write 0xe7158ef thru 0xe71acfb (0x540d bytes) 2024-09-16T11:43:45.421 INFO:teuthology.orchestra.run.smithi070.stdout:1484 read 0xf0934c thru 0xf0c4a7 (0x315c bytes) 2024-09-16T11:43:45.422 INFO:teuthology.orchestra.run.smithi070.stdout:1485 read 0x795e240 thru 0x796920b (0xafcc bytes) 2024-09-16T11:43:45.426 INFO:teuthology.orchestra.run.smithi070.stdout:1486 write 0xe8e1568 thru 0xe8ea1fc (0x8c95 bytes) 2024-09-16T11:43:45.429 INFO:teuthology.orchestra.run.smithi070.stdout:1487 read 0x44338b7 thru 0x4437abb (0x4205 bytes) 2024-09-16T11:43:45.430 INFO:teuthology.orchestra.run.smithi070.stdout:1488 read 0x64da9fa thru 0x64e5868 (0xae6f bytes) 2024-09-16T11:43:45.430 INFO:teuthology.orchestra.run.smithi070.stdout:1490 read 0x4b5b238 thru 0x4b64a71 (0x983a bytes) 2024-09-16T11:43:45.430 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.435 INFO:teuthology.orchestra.run.smithi070.stdout:1492 write 0xee08a9e thru 0xee0acca (0x222d bytes) 2024-09-16T11:43:45.438 INFO:teuthology.orchestra.run.smithi070.stdout:1493 write 0xba42b8a thru 0xba4f2ed (0xc764 bytes) 2024-09-16T11:43:45.440 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.440 INFO:teuthology.orchestra.run.smithi070.stdout:1497 read 0xbb7d2a1 thru 0xbb87677 (0xa3d7 bytes) 2024-09-16T11:43:45.441 INFO:teuthology.orchestra.run.smithi070.stdout:1498 read 0xb2e6a56 thru 0xb2ee1bc (0x7767 bytes) 2024-09-16T11:43:45.441 INFO:teuthology.orchestra.run.smithi070.stdout:1499 write 0x53cbcdd thru 0x53cf500 (0x3824 bytes) 2024-09-16T11:43:45.443 INFO:teuthology.orchestra.run.smithi070.stdout:1500 write 0x34a2b2d thru 0x34a59a8 (0x2e7c bytes) 2024-09-16T11:43:45.444 INFO:teuthology.orchestra.run.smithi070.stdout:1501 write 0xc9677b6 thru 0xc96e51c (0x6d67 bytes) 2024-09-16T11:43:45.460 INFO:teuthology.orchestra.run.smithi070.stdout:1502 trunc from 0xee0accb to 0x419dd57 2024-09-16T11:43:45.474 INFO:teuthology.orchestra.run.smithi070.stdout:1503 read 0x27876f8 thru 0x2793fad (0xc8b6 bytes) 2024-09-16T11:43:45.474 INFO:teuthology.orchestra.run.smithi070.stdout:1504 write 0x2428974 thru 0x242df1d (0x55aa bytes) 2024-09-16T11:43:45.476 INFO:teuthology.orchestra.run.smithi070.stdout:1507 punch from 0x28ffc7f to 0x2904b55, (0x4ed6 bytes) 2024-09-16T11:43:45.476 INFO:teuthology.orchestra.run.smithi070.stdout:1508 write 0x3ac391f thru 0x3acde0a (0xa4ec bytes) 2024-09-16T11:43:45.477 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.490 INFO:teuthology.orchestra.run.smithi070.stdout:1511 write 0xac8976a thru 0xac8ad91 (0x1628 bytes) 2024-09-16T11:43:45.499 INFO:teuthology.orchestra.run.smithi070.stdout:1512 write 0xb49f71d thru 0xb4a4c8d (0x5571 bytes) 2024-09-16T11:43:45.502 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:45.500+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:45.502 INFO:teuthology.orchestra.run.smithi070.stdout:1515 writesame 0x6f78457 thru 0x6f86246 data_size 0x6a(0xddf0 bytes) 2024-09-16T11:43:45.504 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:45.504 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:45.506 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.506 INFO:teuthology.orchestra.run.smithi070.stdout:1517 read 0x67d4bff thru 0x67ddf6c (0x936e bytes) 2024-09-16T11:43:45.512 INFO:teuthology.orchestra.run.smithi070.stdout:1518 trunc from 0xb4a4c8e to 0x60ca365 2024-09-16T11:43:45.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:43:45.522 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.522 INFO:teuthology.orchestra.run.smithi070.stdout:1520 read 0x2b1f879 thru 0x2b223b8 (0x2b40 bytes) 2024-09-16T11:43:45.528 INFO:teuthology.orchestra.run.smithi070.stdout:1521 write 0x62faaa7 thru 0x6302e21 (0x837b bytes) 2024-09-16T11:43:45.530 INFO:teuthology.orchestra.run.smithi070.stdout:1522 punch from 0x2f4301f to 0x2f51842, (0xe823 bytes) 2024-09-16T11:43:45.530 INFO:teuthology.orchestra.run.smithi070.stdout:1523 read 0x4b444c5 thru 0x4b4cd08 (0x8844 bytes) 2024-09-16T11:43:45.531 INFO:teuthology.orchestra.run.smithi070.stdout:1524 write 0x5c0a85a thru 0x5c19f13 (0xf6ba bytes) 2024-09-16T11:43:45.533 INFO:teuthology.orchestra.run.smithi070.stdout:1525 punch from 0x22f43ff to 0x22f778b, (0x338c bytes) 2024-09-16T11:43:45.533 INFO:teuthology.orchestra.run.smithi070.stdout:1526 write 0x4d676e9 thru 0x4d74e6a (0xd782 bytes) 2024-09-16T11:43:45.543 INFO:teuthology.orchestra.run.smithi070.stdout:1528 write 0x85a818a thru 0x85b5224 (0xd09b bytes) 2024-09-16T11:43:45.545 INFO:teuthology.orchestra.run.smithi070.stdout:1529 punch from 0x5ab3fb to 0x5b65d8, (0xb1dd bytes) 2024-09-16T11:43:45.545 INFO:teuthology.orchestra.run.smithi070.stdout:1530 punch from 0x7d34ccf to 0x7d3abb9, (0x5eea bytes) 2024-09-16T11:43:45.545 INFO:teuthology.orchestra.run.smithi070.stdout:1531 write 0xdfa2f4 thru 0xe010ba (0x6dc7 bytes) 2024-09-16T11:43:45.546 INFO:teuthology.orchestra.run.smithi070.stdout:1533 writesame 0x19d6b90 thru 0x19dd379 data_size 0x2f(0x67ea bytes) 2024-09-16T11:43:45.559 INFO:teuthology.orchestra.run.smithi070.stdout:1534 write 0xce2c83c thru 0xce3abdc (0xe3a1 bytes) 2024-09-16T11:43:45.562 INFO:teuthology.orchestra.run.smithi070.stdout:1537 writesame 0x109d018 thru 0x10a1dec data_size 0x5(0x4dd5 bytes) 2024-09-16T11:43:45.566 INFO:teuthology.orchestra.run.smithi070.stdout:1538 read 0x55e8fe2 thru 0x55ebbea (0x2c09 bytes) 2024-09-16T11:43:45.574 INFO:teuthology.orchestra.run.smithi070.stdout:1540 write 0xe212a18 thru 0xe2154b5 (0x2a9e bytes) 2024-09-16T11:43:45.576 INFO:teuthology.orchestra.run.smithi070.stdout:1541 write 0x871b59b thru 0x87252fe (0x9d64 bytes) 2024-09-16T11:43:45.578 INFO:teuthology.orchestra.run.smithi070.stdout:1542 read 0x7d6c137 thru 0x7d760d2 (0x9f9c bytes) 2024-09-16T11:43:45.578 INFO:teuthology.orchestra.run.smithi070.stdout:1543 writesame 0x4af51e thru 0x4baa30 data_size 0x49(0xb513 bytes) 2024-09-16T11:43:45.579 INFO:teuthology.orchestra.run.smithi070.stdout:1545 write 0x1909376 thru 0x190e97c (0x5607 bytes) 2024-09-16T11:43:45.581 INFO:teuthology.orchestra.run.smithi070.stdout:1546 write 0x5419c94 thru 0x54212f7 (0x7664 bytes) 2024-09-16T11:43:45.584 INFO:teuthology.orchestra.run.smithi070.stdout:1547 read 0x524637c thru 0x5249429 (0x30ae bytes) 2024-09-16T11:43:45.584 INFO:teuthology.orchestra.run.smithi070.stdout:1549 punch from 0xc15892a to 0xc1688bd, (0xff93 bytes) 2024-09-16T11:43:45.584 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.584 INFO:teuthology.orchestra.run.smithi070.stdout:1551 punch from 0xb577c4e to 0xb5876ef, (0xfaa1 bytes) 2024-09-16T11:43:45.584 INFO:teuthology.orchestra.run.smithi070.stdout:1552 writesame 0x67b6dfa thru 0x67bac17 data_size 0x1(0x3e1e bytes) 2024-09-16T11:43:45.592 INFO:teuthology.orchestra.run.smithi070.stdout:1553 write 0x465fc2e thru 0x4669309 (0x96dc bytes) 2024-09-16T11:43:45.593 INFO:teuthology.orchestra.run.smithi070.stdout:1555 read 0x7827e4b thru 0x782a050 (0x2206 bytes) 2024-09-16T11:43:45.594 INFO:teuthology.orchestra.run.smithi070.stdout:1556 punch from 0x4e4311 to 0x4f3033, (0xed22 bytes) 2024-09-16T11:43:45.594 INFO:teuthology.orchestra.run.smithi070.stdout:1557 write 0x25fb2cd thru 0x25ff22d (0x3f61 bytes) 2024-09-16T11:43:45.595 INFO:teuthology.orchestra.run.smithi070.stdout:1560 trunc from 0xe2154b6 to 0xe806927 2024-09-16T11:43:45.600 INFO:teuthology.orchestra.run.smithi070.stdout:1561 read 0x4ff3f8a thru 0x5000b86 (0xcbfd bytes) 2024-09-16T11:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout:1564 writesame 0x2ab8e24 thru 0x2abfb3d data_size 0x39(0x6d1a bytes) 2024-09-16T11:43:45.602 INFO:teuthology.orchestra.run.smithi070.stdout:1565 write 0x914ee41 thru 0x9153148 (0x4308 bytes) 2024-09-16T11:43:45.603 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:45.601+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout:1566 write 0x3318a3a thru 0x331e2c0 (0x5887 bytes) 2024-09-16T11:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout:1568 writesame 0xad2c786 thru 0xad31a56 data_size 0x25(0x52d1 bytes) 2024-09-16T11:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout:1569 read 0xd7126c5 thru 0xd71a712 (0x804e bytes) 2024-09-16T11:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout:1570 punch from 0x5445eb4 to 0x54484b1, (0x25fd bytes) 2024-09-16T11:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout:1571 read 0x3bc0dd7 thru 0x3bccc26 (0xbe50 bytes) 2024-09-16T11:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout:1572 read 0x21d6240 thru 0x21e207e (0xbe3f bytes) 2024-09-16T11:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout:1573 write 0x1668411 thru 0x166cd8b (0x497b bytes) 2024-09-16T11:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout:1574 punch from 0x5f7027e to 0x5f7c4de, (0xc260 bytes) 2024-09-16T11:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:1575 trunc from 0xe806927 to 0x75d1695 2024-09-16T11:43:45.635 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:45.635 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:45.643 INFO:teuthology.orchestra.run.smithi070.stdout:1576 write 0xe3b4348 thru 0xe3be79d (0xa456 bytes) 2024-09-16T11:43:45.646 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:43:45.647 INFO:teuthology.orchestra.run.smithi070.stdout:1577 read 0xc49fed3 thru 0xc4a241b (0x2549 bytes) 2024-09-16T11:43:45.658 INFO:teuthology.orchestra.run.smithi070.stdout:1578 trunc from 0xe3be79e to 0x3ef5941 2024-09-16T11:43:45.672 INFO:teuthology.orchestra.run.smithi070.stdout:1581 trunc from 0x3ef5941 to 0x2fb86e5 2024-09-16T11:43:45.694 INFO:teuthology.orchestra.run.smithi070.stdout:1582 write 0xa429f8b thru 0xa4396d4 (0xf74a bytes) 2024-09-16T11:43:45.697 INFO:teuthology.orchestra.run.smithi070.stdout:1583 writesame 0x6fd4492 thru 0x6fddf14 data_size 0x2d(0x9a83 bytes) 2024-09-16T11:43:45.700 INFO:teuthology.orchestra.run.smithi070.stdout:1584 punch from 0x80ac9c8 to 0x80b68d0, (0x9f08 bytes) 2024-09-16T11:43:45.700 INFO:teuthology.orchestra.run.smithi070.stdout:1585 writesame 0x61bda9c thru 0x61c9e55 data_size 0x15(0xc3ba bytes) 2024-09-16T11:43:45.703 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:45.701+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:1586 writesame 0xb6f1202 thru 0xb6f7849 data_size 0xc(0x6648 bytes) 2024-09-16T11:43:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.718 INFO:teuthology.orchestra.run.smithi070.stdout:1589 write 0xd2e631f thru 0xd2f14b6 (0xb198 bytes) 2024-09-16T11:43:45.721 INFO:teuthology.orchestra.run.smithi070.stdout:1591 read 0x5b82f63 thru 0x5b8e647 (0xb6e5 bytes) 2024-09-16T11:43:45.721 INFO:teuthology.orchestra.run.smithi070.stdout:1595 read 0x38ee3fa thru 0x38eec0d (0x814 bytes) 2024-09-16T11:43:45.722 INFO:teuthology.orchestra.run.smithi070.stdout:1597 punch from 0x9f4f82a to 0x9f5c043, (0xc819 bytes) 2024-09-16T11:43:45.722 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.722 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.722 INFO:teuthology.orchestra.run.smithi070.stdout:1602 writesame 0x8793532 thru 0x8795b6d data_size 0x2(0x263c bytes) 2024-09-16T11:43:45.726 INFO:teuthology.orchestra.run.smithi070.stdout:skipping zero size write 2024-09-16T11:43:45.726 INFO:teuthology.orchestra.run.smithi070.stdout:1604 writesame 0x765e567 thru 0x7668438 data_size 0x1d(0x9ed2 bytes) 2024-09-16T11:43:45.743 INFO:teuthology.orchestra.run.smithi070.stdout:1605 trunc from 0xd2f14b7 to 0x1c4b4f4 2024-09-16T11:43:45.751 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:45.751 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:45.754 INFO:teuthology.orchestra.run.smithi070.stdout:1607 read 0x4c945a thru 0x4cdb1a (0x46c1 bytes) 2024-09-16T11:43:45.754 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.754 INFO:teuthology.orchestra.run.smithi070.stdout:1609 read 0x107c2e9 thru 0x1086419 (0xa131 bytes) 2024-09-16T11:43:45.760 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:43:45.765 INFO:teuthology.orchestra.run.smithi070.stdout:1610 writesame 0x62fc127 thru 0x6304d2c data_size 0x1(0x8c06 bytes) 2024-09-16T11:43:45.765 INFO:teuthology.orchestra.run.smithi070.stdout:1611 read 0x2024e2b thru 0x202e833 (0x9a09 bytes) 2024-09-16T11:43:45.765 INFO:teuthology.orchestra.run.smithi070.stdout:1613 write 0x5675774 thru 0x567ed92 (0x961f bytes) 2024-09-16T11:43:45.772 INFO:teuthology.orchestra.run.smithi070.stdout:1615 trunc from 0x6304d2d to 0x3167c09 2024-09-16T11:43:45.782 INFO:teuthology.orchestra.run.smithi070.stdout:1616 read 0x3161f44 thru 0x31622fb (0x3b8 bytes) 2024-09-16T11:43:45.783 INFO:teuthology.orchestra.run.smithi070.stdout:1621 punch from 0x2508069 to 0x250d472, (0x5409 bytes) 2024-09-16T11:43:45.788 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:43:45.789 INFO:teuthology.orchestra.run.smithi070.stdout:1623 write 0x3de1c6f thru 0x3ded7ec (0xbb7e bytes) 2024-09-16T11:43:45.791 INFO:teuthology.orchestra.run.smithi070.stdout:1626 read 0x2fef525 thru 0x2ff39e7 (0x44c3 bytes) 2024-09-16T11:43:45.799 INFO:teuthology.orchestra.run.smithi070.stdout:1627 write 0x5be1b4d thru 0x5beb39f (0x9853 bytes) 2024-09-16T11:43:45.802 INFO:teuthology.orchestra.run.smithi070.stdout:1628 write 0x2936a88 thru 0x293b76d (0x4ce6 bytes) 2024-09-16T11:43:45.804 INFO:teuthology.orchestra.run.smithi070.stdout:1629 read 0x2cd834a thru 0x2cdd8cd (0x5584 bytes) 2024-09-16T11:43:45.804 INFO:teuthology.orchestra.run.smithi070.stdout:1631 read 0x1372bc2 thru 0x1377bf2 (0x5031 bytes) 2024-09-16T11:43:45.804 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:45.802+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:45.804 INFO:teuthology.orchestra.run.smithi070.stdout:1632 read 0xe9f557 thru 0xeac22e (0xccd8 bytes) 2024-09-16T11:43:45.814 INFO:teuthology.orchestra.run.smithi070.stdout:1633 trunc from 0x5beb3a0 to 0xd3ec249 2024-09-16T11:43:45.819 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T11:43:45.826 INFO:teuthology.orchestra.run.smithi070.stdout:1634 trunc from 0xd3ec249 to 0x943c643 2024-09-16T11:43:45.831 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.832 INFO:teuthology.orchestra.run.smithi070.stdout:1637 writesame 0x571d765 thru 0x57260f4 data_size 0x47(0x8990 bytes) 2024-09-16T11:43:45.837 INFO:teuthology.orchestra.run.smithi070.stdout:1638 write 0xea3edd thru 0xeb0f16 (0xd03a bytes) 2024-09-16T11:43:45.838 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.838 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.845 INFO:teuthology.orchestra.run.smithi070.stdout:1641 trunc from 0x943c643 to 0x40a2c86 2024-09-16T11:43:45.857 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.858 INFO:teuthology.orchestra.run.smithi070.stdout:1643 punch from 0x311cb25 to 0x3126dc4, (0xa29f bytes) 2024-09-16T11:43:45.858 INFO:teuthology.orchestra.run.smithi070.stdout:1645 read 0x10c8555 thru 0x10d193f (0x93eb bytes) 2024-09-16T11:43:45.858 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.858 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.866 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:45.866 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:45.871 INFO:teuthology.orchestra.run.smithi070.stdout:1649 write 0xb16161d thru 0xb1632b2 (0x1c96 bytes) 2024-09-16T11:43:45.874 INFO:teuthology.orchestra.run.smithi070.stdout:1650 write 0x9bdc97c thru 0x9bec6b2 (0xfd37 bytes) 2024-09-16T11:43:45.876 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:43:45.879 INFO:teuthology.orchestra.run.smithi070.stdout:1651 trunc from 0xb1632b3 to 0xa064d20 2024-09-16T11:43:45.895 INFO:teuthology.orchestra.run.smithi070.stdout:1652 trunc from 0xa064d20 to 0x1ba1468 2024-09-16T11:43:45.904 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:45.902+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:45.912 INFO:teuthology.orchestra.run.smithi070.stdout:1654 write 0x20ce9da thru 0x20d0f12 (0x2539 bytes) 2024-09-16T11:43:45.913 INFO:teuthology.orchestra.run.smithi070.stdout:1655 read 0x1aa594a thru 0x1aaa458 (0x4b0f bytes) 2024-09-16T11:43:45.914 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.914 INFO:teuthology.orchestra.run.smithi070.stdout:1658 read 0xebbd6c thru 0xec263e (0x68d3 bytes) 2024-09-16T11:43:45.914 INFO:teuthology.orchestra.run.smithi070.stdout:1660 read 0xbca28f thru 0xbd021e (0x5f90 bytes) 2024-09-16T11:43:45.926 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:43:45.926 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:43:45.932 INFO:teuthology.orchestra.run.smithi070.stdout:1661 writesame 0xc12fa60 thru 0xc134a01 data_size 0x1(0x4fa2 bytes) 2024-09-16T11:43:45.932 INFO:teuthology.orchestra.run.smithi070.stdout:1663 read 0x201005d thru 0x201ed24 (0xecc8 bytes) 2024-09-16T11:43:45.932 INFO:teuthology.orchestra.run.smithi070.stdout:1664 read 0x22d988c thru 0x22e6d39 (0xd4ae bytes) 2024-09-16T11:43:45.932 INFO:teuthology.orchestra.run.smithi070.stdout:1665 read 0x428c712 thru 0x4292e89 (0x6778 bytes) 2024-09-16T11:43:45.932 INFO:teuthology.orchestra.run.smithi070.stdout:1666 write 0x50669e thru 0x50bdc7 (0x572a bytes) 2024-09-16T11:43:45.934 INFO:teuthology.orchestra.run.smithi070.stdout:1667 write 0xaa925d thru 0xab6626 (0xd3ca bytes) 2024-09-16T11:43:45.936 INFO:teuthology.orchestra.run.smithi070.stdout:1668 read 0xa1597b4 thru 0xa168b51 (0xf39e bytes) 2024-09-16T11:43:45.936 INFO:teuthology.orchestra.run.smithi070.stdout:1669 read 0x581ef83 thru 0x58239d0 (0x4a4e bytes) 2024-09-16T11:43:45.936 INFO:teuthology.orchestra.run.smithi070.stdout:1670 punch from 0x4c65921 to 0x4c6fad1, (0xa1b0 bytes) 2024-09-16T11:43:45.936 INFO:teuthology.orchestra.run.smithi070.stdout:1671 write 0x82b2b88 thru 0x82b3558 (0x9d1 bytes) 2024-09-16T11:43:45.938 INFO:teuthology.orchestra.run.smithi070.stdout:1673 punch from 0x7bc3e78 to 0x7bcc788, (0x8910 bytes) 2024-09-16T11:43:45.942 INFO:teuthology.orchestra.run.smithi070.stdout:1674 trunc from 0xc134a02 to 0x91228f3 2024-09-16T11:43:45.943 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:43:45.944 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:43:45.956 INFO:teuthology.orchestra.run.smithi070.stdout:1676 trunc from 0x91228f3 to 0x1bf4f5a 2024-09-16T11:43:45.960 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:43:45.961 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:43:45.966 INFO:teuthology.orchestra.run.smithi070.stdout:1677 punch from 0xbfeec6 to 0xc03bed, (0x4d27 bytes) 2024-09-16T11:43:45.966 INFO:teuthology.orchestra.run.smithi070.stdout:1678 write 0xec11b4 thru 0xece2e7 (0xd134 bytes) 2024-09-16T11:43:45.977 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:45.977 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:45.977 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:43:45.978 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:43:45.982 INFO:teuthology.orchestra.run.smithi070.stdout:1679 write 0x92edea3 thru 0x92f959b (0xb6f9 bytes) 2024-09-16T11:43:45.985 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:45.986 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:43:45.988 INFO:teuthology.orchestra.run.smithi070.stdout:1681 trunc from 0x92f959c to 0x6b73b45 2024-09-16T11:43:45.995 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:43:45.995 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:43:45.997 INFO:teuthology.orchestra.run.smithi070.stdout:1682 read 0x205000f thru 0x2057833 (0x7825 bytes) 2024-09-16T11:43:45.997 INFO:teuthology.orchestra.run.smithi070.stdout:1683 punch from 0x54cf358 to 0x54cf545, (0x1ed bytes) 2024-09-16T11:43:45.997 INFO:teuthology.orchestra.run.smithi070.stdout:1684 writesame 0x483e106 thru 0x4843cf7 data_size 0x3(0x5bf2 bytes) 2024-09-16T11:43:46.002 INFO:teuthology.orchestra.run.smithi070.stdout:1685 write 0x12c9b01 thru 0x12d8147 (0xe647 bytes) 2024-09-16T11:43:46.004 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:46.005 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:46.003+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:46.012 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:43:46.012 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:43:46.015 INFO:teuthology.orchestra.run.smithi070.stdout:1687 write 0xbd2ad88 thru 0xbd2fb7c (0x4df5 bytes) 2024-09-16T11:43:46.018 INFO:teuthology.orchestra.run.smithi070.stdout:1688 writesame 0x37faa1b thru 0x37fb1ae data_size 0xa(0x794 bytes) 2024-09-16T11:43:46.020 INFO:teuthology.orchestra.run.smithi070.stdout:1689 write 0x7c2fc4c thru 0x7c35421 (0x57d6 bytes) 2024-09-16T11:43:46.023 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:46.023 INFO:teuthology.orchestra.run.smithi070.stdout:1694 read 0x2e85e7f thru 0x2e8f1dd (0x935f bytes) 2024-09-16T11:43:46.023 INFO:teuthology.orchestra.run.smithi070.stdout:1695 write 0xa3666da thru 0xa36b741 (0x5068 bytes) 2024-09-16T11:43:46.026 INFO:teuthology.orchestra.run.smithi070.stdout:1696 write 0x186e47b thru 0x1878f7a (0xab00 bytes) 2024-09-16T11:43:46.028 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:43:46.028 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:43:46.029 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:46.029 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:46.029 INFO:teuthology.orchestra.run.smithi070.stdout:1700 writesame 0xa24a0db thru 0xa256c53 data_size 0x3(0xcb79 bytes) 2024-09-16T11:43:46.036 INFO:teuthology.orchestra.run.smithi070.stdout:1701 punch from 0x109238a to 0x109d280, (0xaef6 bytes) 2024-09-16T11:43:46.037 INFO:teuthology.orchestra.run.smithi070.stdout:1704 write 0xc4c48b thru 0xc5ae22 (0xe998 bytes) 2024-09-16T11:43:46.038 INFO:teuthology.orchestra.run.smithi070.stdout:1706 write 0x56db9df thru 0x56e9000 (0xd622 bytes) 2024-09-16T11:43:46.040 INFO:teuthology.orchestra.run.smithi070.stdout:1707 read 0x7359e7f thru 0x7360e09 (0x6f8b bytes) 2024-09-16T11:43:46.041 INFO:teuthology.orchestra.run.smithi070.stdout:1708 punch from 0x70b78a8 to 0x70bcdc4, (0x551c bytes) 2024-09-16T11:43:46.041 INFO:teuthology.orchestra.run.smithi070.stdout:1710 read 0x532e748 thru 0x533a984 (0xc23d bytes) 2024-09-16T11:43:46.041 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:46.041 INFO:teuthology.orchestra.run.smithi070.stdout:1712 writesame 0x77ee1e thru 0x78414c data_size 0x1(0x532f bytes) 2024-09-16T11:43:46.046 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:43:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:43:46.049 INFO:teuthology.orchestra.run.smithi070.stdout:1716 write 0x71383eb thru 0x713f13c (0x6d52 bytes) 2024-09-16T11:43:46.052 INFO:teuthology.orchestra.run.smithi070.stdout:1717 read 0x83d3720 thru 0x83d3d9f (0x680 bytes) 2024-09-16T11:43:46.053 INFO:teuthology.orchestra.run.smithi070.stdout:1722 writesame 0x4889a1f thru 0x489964d data_size 0x1(0xfc2f bytes) 2024-09-16T11:43:46.082 INFO:teuthology.orchestra.run.smithi070.stdout:1723 writesame 0x6c08363 thru 0x6c141e5 data_size 0x3(0xbe83 bytes) 2024-09-16T11:43:46.087 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:46.087 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:46.088 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:43:46.088 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:46.097 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:43:46.098 INFO:teuthology.orchestra.run.smithi070.stdout:1725 trunc from 0xbd2fb7d to 0x681f48b 2024-09-16T11:43:46.109 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:46.109 INFO:teuthology.orchestra.run.smithi070.stdout:1727 punch from 0x4ca85be to 0x4cb1eb7, (0x98f9 bytes) 2024-09-16T11:43:46.109 INFO:teuthology.orchestra.run.smithi070.stdout:1728 read 0x3979b thru 0x3a321 (0xb87 bytes) 2024-09-16T11:43:46.115 INFO:teuthology.orchestra.run.smithi070.stdout:1729 trunc from 0x681f48b to 0xb563359 2024-09-16T11:43:46.120 INFO:teuthology.orchestra.run.smithi070.stdout:1730 writesame 0x54f4447 thru 0x550363c data_size 0x1(0xf1f6 bytes) 2024-09-16T11:43:46.141 INFO:teuthology.orchestra.run.smithi070.stdout:1732 write 0x32dcd44 thru 0x32ec867 (0xfb24 bytes) 2024-09-16T11:43:46.141 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:46.140+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:46.143 INFO:teuthology.orchestra.run.smithi070.stdout:1733 read 0xe97c9 thru 0xf0e00 (0x7638 bytes) 2024-09-16T11:43:46.143 INFO:teuthology.orchestra.run.smithi070.stdout:1734 punch from 0xa6eec94 to 0xa6f19df, (0x2d4b bytes) 2024-09-16T11:43:46.143 INFO:teuthology.orchestra.run.smithi070.stdout:1735 read 0x18ecd0a thru 0x18f274e (0x5a45 bytes) 2024-09-16T11:43:46.143 INFO:teuthology.orchestra.run.smithi070.stdout:1737 read 0x518d8d9 thru 0x5199200 (0xb928 bytes) 2024-09-16T11:43:46.155 INFO:teuthology.orchestra.run.smithi070.stdout:1739 trunc from 0xb563359 to 0x1e28f78 2024-09-16T11:43:46.169 INFO:teuthology.orchestra.run.smithi070.stdout:1741 read 0x7c1a1c thru 0x7d03f2 (0xe9d7 bytes) 2024-09-16T11:43:46.169 INFO:teuthology.orchestra.run.smithi070.stdout:1744 read 0xad5c0f thru 0xada709 (0x4afb bytes) 2024-09-16T11:43:46.177 INFO:teuthology.orchestra.run.smithi070.stdout:1746 write 0x4158892 thru 0x4166f99 (0xe708 bytes) 2024-09-16T11:43:46.180 INFO:teuthology.orchestra.run.smithi070.stdout:1747 read 0x35db36b thru 0x35dbd37 (0x9cd bytes) 2024-09-16T11:43:46.180 INFO:teuthology.orchestra.run.smithi070.stdout:1748 punch from 0x209fcef to 0x20a3fe3, (0x42f4 bytes) 2024-09-16T11:43:46.180 INFO:teuthology.orchestra.run.smithi070.stdout:1749 punch from 0x3501857 to 0x350e59f, (0xcd48 bytes) 2024-09-16T11:43:46.180 INFO:teuthology.orchestra.run.smithi070.stdout:1750 read 0x29bae0e thru 0x29ca6bf (0xf8b2 bytes) 2024-09-16T11:43:46.195 INFO:teuthology.orchestra.run.smithi070.stdout:1753 write 0xbce5147 thru 0xbcf25b6 (0xd470 bytes) 2024-09-16T11:43:46.198 INFO:teuthology.orchestra.run.smithi070.stdout:1754 read 0x428a498 thru 0x429913d (0xeca6 bytes) 2024-09-16T11:43:46.198 INFO:teuthology.orchestra.run.smithi070.stdout:1755 read 0x3c707b6 thru 0x3c7aa60 (0xa2ab bytes) 2024-09-16T11:43:46.198 INFO:teuthology.orchestra.run.smithi070.stdout:1756 write 0x2952af9 thru 0x2959a02 (0x6f0a bytes) 2024-09-16T11:43:46.200 INFO:teuthology.orchestra.run.smithi070.stdout:1757 write 0x9892c4d thru 0x9894fb0 (0x2364 bytes) 2024-09-16T11:43:46.200 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:46.200 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:46.201 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:46.202 INFO:teuthology.orchestra.run.smithi070.stdout:1759 read 0x6f1575e thru 0x6f1f76f (0xa012 bytes) 2024-09-16T11:43:46.202 INFO:teuthology.orchestra.run.smithi070.stdout:1761 writesame 0x3d1ffba thru 0x3d2f224 data_size 0x1(0xf26b bytes) 2024-09-16T11:43:46.206 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:46.204+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:46.210 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:43:46.231 INFO:teuthology.orchestra.run.smithi070.stdout:1764 clone 8 order 21 su 131072 sc 7 2024-09-16T11:43:46.307 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:46.306+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:46.350 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:43:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:43:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:43:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2119 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2375680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864670+0000", 2024-09-16T11:43:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4833092380000004, 2024-09-16T11:43:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013064175000000001, 2024-09-16T11:43:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.351 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2119 2024-09-16T11:43:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864670+0000", 2024-09-16T11:43:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864670+0000", 2024-09-16T11:43:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864670+0000", 2024-09-16T11:43:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:43:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864672+0000", 2024-09-16T11:43:46.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.691e-06 2024-09-16T11:43:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864677+0000", 2024-09-16T11:43:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6860000000000003e-06 2024-09-16T11:43:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864681+0000", 2024-09-16T11:43:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-06 2024-09-16T11:43:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877539+0000", 2024-09-16T11:43:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012858748999999999 2024-09-16T11:43:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877632+0000", 2024-09-16T11:43:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2367999999999994e-05 2024-09-16T11:43:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877734+0000", 2024-09-16T11:43:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102 2024-09-16T11:43:46.355 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2120 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864685+0000", 2024-09-16T11:43:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4832942500000001, 2024-09-16T11:43:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013171611999999999, 2024-09-16T11:43:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2120 2024-09-16T11:43:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864685+0000", 2024-09-16T11:43:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864685+0000", 2024-09-16T11:43:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864685+0000", 2024-09-16T11:43:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-16T11:43:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864692+0000", 2024-09-16T11:43:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5039999999999999e-06 2024-09-16T11:43:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864697+0000", 2024-09-16T11:43:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.699e-06 2024-09-16T11:43:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864701+0000", 2024-09-16T11:43:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6009999999999999e-06 2024-09-16T11:43:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877756+0000", 2024-09-16T11:43:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0130554 2024-09-16T11:43:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877801+0000", 2024-09-16T11:43:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5157000000000002e-05 2024-09-16T11:43:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877856+0000", 2024-09-16T11:43:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4910000000000001e-05 2024-09-16T11:43:46.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2121 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2818048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864708+0000", 2024-09-16T11:43:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4832712079999997, 2024-09-16T11:43:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013354938, 2024-09-16T11:43:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2121 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864708+0000", 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864708+0000", 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864708+0000", 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864710+0000", 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864716+0000", 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7289999999999997e-06 2024-09-16T11:43:46.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864720+0000", 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9990000000000002e-06 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877883+0000", 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013162903 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.877969+0000", 2024-09-16T11:43:46.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6027999999999997e-05 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878063+0000", 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4086999999999996e-05 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2122 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864723+0000", 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4832559590000001, 2024-09-16T11:43:46.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013470271000000001, 2024-09-16T11:43:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2122 2024-09-16T11:43:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864723+0000", 2024-09-16T11:43:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864723+0000", 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864723+0000", 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6199999999999999e-07 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864725+0000", 2024-09-16T11:43:46.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5960000000000001e-06 2024-09-16T11:43:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864730+0000", 2024-09-16T11:43:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1950000000000002e-06 2024-09-16T11:43:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864733+0000", 2024-09-16T11:43:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-06 2024-09-16T11:43:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878093+0000", 2024-09-16T11:43:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013359473 2024-09-16T11:43:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878140+0000", 2024-09-16T11:43:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6692e-05 2024-09-16T11:43:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878193+0000", 2024-09-16T11:43:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3683000000000001e-05 2024-09-16T11:43:46.369 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2124 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2326528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864738+0000", 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4832411649999999, 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013679180000000001, 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2124 2024-09-16T11:43:46.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864738+0000", 2024-09-16T11:43:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864738+0000", 2024-09-16T11:43:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864738+0000", 2024-09-16T11:43:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:43:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864740+0000", 2024-09-16T11:43:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5990000000000001e-06 2024-09-16T11:43:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864745+0000", 2024-09-16T11:43:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5600000000000001e-06 2024-09-16T11:43:46.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864749+0000", 2024-09-16T11:43:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2590000000000001e-06 2024-09-16T11:43:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878230+0000", 2024-09-16T11:43:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013481215 2024-09-16T11:43:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878314+0000", 2024-09-16T11:43:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4037999999999997e-05 2024-09-16T11:43:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878417+0000", 2024-09-16T11:43:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010315900000000001 2024-09-16T11:43:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.374 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2125 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2523136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864752+0000", 2024-09-16T11:43:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.483227061, 2024-09-16T11:43:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013813690999999999, 2024-09-16T11:43:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2125 2024-09-16T11:43:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864752+0000", 2024-09-16T11:43:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864752+0000", 2024-09-16T11:43:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864752+0000", 2024-09-16T11:43:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:43:46.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864754+0000", 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864760+0000", 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5119999999999997e-06 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864764+0000", 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5779999999999999e-06 2024-09-16T11:43:46.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878448+0000", 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013683749 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878496+0000", 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8028999999999997e-05 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878566+0000", 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9900000000000005e-05 2024-09-16T11:43:46.378 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2126 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2719744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.864767+0000", 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4832118860000003, 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013991063999999999, 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.379 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2126 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864767+0000", 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864767+0000", 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864767+0000", 2024-09-16T11:43:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1800000000000002e-07 2024-09-16T11:43:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864769+0000", 2024-09-16T11:43:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.53e-06 2024-09-16T11:43:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864772+0000", 2024-09-16T11:43:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4580000000000002e-06 2024-09-16T11:43:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.864774+0000", 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.979e-06 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878598+0000", 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013823821999999999 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878673+0000", 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4476999999999999e-05 2024-09-16T11:43:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878758+0000", 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-05 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2182 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 65536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.865997+0000", 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4819821019999999, 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012930596000000001, 2024-09-16T11:43:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2182 2024-09-16T11:43:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865997+0000", 2024-09-16T11:43:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865997+0000", 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.865998+0000", 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866001+0000", 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7220000000000001e-06 2024-09-16T11:43:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866013+0000", 2024-09-16T11:43:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.188e-05 2024-09-16T11:43:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866026+0000", 2024-09-16T11:43:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2938999999999999e-05 2024-09-16T11:43:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878795+0000", 2024-09-16T11:43:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012769024 2024-09-16T11:43:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878864+0000", 2024-09-16T11:43:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8427000000000001e-05 2024-09-16T11:43:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878927+0000", 2024-09-16T11:43:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3746999999999998e-05 2024-09-16T11:43:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2183 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 262144~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866033+0000", 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4819454849999998, 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013099107, 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2183 2024-09-16T11:43:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866033+0000", 2024-09-16T11:43:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866033+0000", 2024-09-16T11:43:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866034+0000", 2024-09-16T11:43:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:43:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866036+0000", 2024-09-16T11:43:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:43:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866047+0000", 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1083999999999999e-05 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866062+0000", 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5156e-05 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.878961+0000", 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012898687000000001 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879036+0000", 2024-09-16T11:43:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4540000000000001e-05 2024-09-16T11:43:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879133+0000", 2024-09-16T11:43:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6887999999999998e-05 2024-09-16T11:43:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2184 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 507904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866067+0000", 2024-09-16T11:43:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4819114759999996, 2024-09-16T11:43:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013274744, 2024-09-16T11:43:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2184 2024-09-16T11:43:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866067+0000", 2024-09-16T11:43:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866067+0000", 2024-09-16T11:43:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866068+0000", 2024-09-16T11:43:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:43:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866070+0000", 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866076+0000", 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1809999999999998e-06 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866080+0000", 2024-09-16T11:43:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8279999999999996e-06 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879170+0000", 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013089747000000001 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879252+0000", 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2279000000000003e-05 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879342+0000", 2024-09-16T11:43:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0195e-05 2024-09-16T11:43:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2185 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 704512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866084+0000", 2024-09-16T11:43:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.48189511, 2024-09-16T11:43:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013513447, 2024-09-16T11:43:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2185 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866084+0000", 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866084+0000", 2024-09-16T11:43:46.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866084+0000", 2024-09-16T11:43:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.39e-07 2024-09-16T11:43:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866086+0000", 2024-09-16T11:43:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6789999999999999e-06 2024-09-16T11:43:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866091+0000", 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.626e-06 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866095+0000", 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879406+0000", 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013311523 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879497+0000", 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0507999999999995e-05 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879597+0000", 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010027900000000001 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2186 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 901120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866100+0000", 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4818790750000002, 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013635298000000001, 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2186 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866100+0000", 2024-09-16T11:43:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866100+0000", 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866100+0000", 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.15e-07 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866109+0000", 2024-09-16T11:43:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8189999999999998e-06 2024-09-16T11:43:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866115+0000", 2024-09-16T11:43:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2770000000000002e-06 2024-09-16T11:43:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866121+0000", 2024-09-16T11:43:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1279999999999996e-06 2024-09-16T11:43:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879618+0000", 2024-09-16T11:43:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013496869 2024-09-16T11:43:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879676+0000", 2024-09-16T11:43:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8075000000000002e-05 2024-09-16T11:43:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879735+0000", 2024-09-16T11:43:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8715000000000003e-05 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2187 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866125+0000", 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4818534110000003, 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013740278999999999, 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2187 2024-09-16T11:43:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866125+0000", 2024-09-16T11:43:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866125+0000", 2024-09-16T11:43:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866126+0000", 2024-09-16T11:43:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.08e-07 2024-09-16T11:43:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866128+0000", 2024-09-16T11:43:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6160000000000001e-06 2024-09-16T11:43:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866145+0000", 2024-09-16T11:43:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.747e-05 2024-09-16T11:43:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866161+0000", 2024-09-16T11:43:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6382e-05 2024-09-16T11:43:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879767+0000", 2024-09-16T11:43:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013605894 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879813+0000", 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5399999999999999e-05 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879866+0000", 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3109000000000003e-05 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2188 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1540096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866166+0000", 2024-09-16T11:43:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4818130759999999, 2024-09-16T11:43:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013814610999999999, 2024-09-16T11:43:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2188 2024-09-16T11:43:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866166+0000", 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866166+0000", 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866166+0000", 2024-09-16T11:43:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:43:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866181+0000", 2024-09-16T11:43:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.489e-05 2024-09-16T11:43:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866185+0000", 2024-09-16T11:43:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.386e-06 2024-09-16T11:43:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866188+0000", 2024-09-16T11:43:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.469e-06 2024-09-16T11:43:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879888+0000", 2024-09-16T11:43:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013700273000000001 2024-09-16T11:43:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879932+0000", 2024-09-16T11:43:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4039000000000003e-05 2024-09-16T11:43:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.879980+0000", 2024-09-16T11:43:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8186000000000002e-05 2024-09-16T11:43:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2189 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866191+0000", 2024-09-16T11:43:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4817883959999998, 2024-09-16T11:43:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013900292999999999, 2024-09-16T11:43:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2189 2024-09-16T11:43:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866191+0000", 2024-09-16T11:43:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866191+0000", 2024-09-16T11:43:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866191+0000", 2024-09-16T11:43:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1699999999999999e-07 2024-09-16T11:43:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866193+0000", 2024-09-16T11:43:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3410000000000001e-06 2024-09-16T11:43:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866196+0000", 2024-09-16T11:43:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9840000000000001e-06 2024-09-16T11:43:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866198+0000", 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8369999999999999e-06 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880001+0000", 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013803426000000001 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880042+0000", 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0410000000000001e-05 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880091+0000", 2024-09-16T11:43:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8977999999999998e-05 2024-09-16T11:43:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2190 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 1933312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866200+0000", 2024-09-16T11:43:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4817791180000004, 2024-09-16T11:43:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013997562, 2024-09-16T11:43:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2190 2024-09-16T11:43:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866200+0000", 2024-09-16T11:43:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866200+0000", 2024-09-16T11:43:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866200+0000", 2024-09-16T11:43:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1899999999999999e-07 2024-09-16T11:43:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866201+0000", 2024-09-16T11:43:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0249999999999999e-06 2024-09-16T11:43:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866204+0000", 2024-09-16T11:43:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.858e-06 2024-09-16T11:43:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866206+0000", 2024-09-16T11:43:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:43:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880111+0000", 2024-09-16T11:43:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013905463999999999 2024-09-16T11:43:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880151+0000", 2024-09-16T11:43:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0182999999999998e-05 2024-09-16T11:43:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880197+0000", 2024-09-16T11:43:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6050000000000001e-05 2024-09-16T11:43:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2191 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2129920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866207+0000", 2024-09-16T11:43:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4817714640000004, 2024-09-16T11:43:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014099525, 2024-09-16T11:43:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2191 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866207+0000", 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866207+0000", 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866208+0000", 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866209+0000", 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.519e-06 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866212+0000", 2024-09-16T11:43:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8210000000000002e-06 2024-09-16T11:43:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866218+0000", 2024-09-16T11:43:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5280000000000003e-06 2024-09-16T11:43:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880218+0000", 2024-09-16T11:43:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014000383999999999 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880259+0000", 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0815000000000001e-05 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880307+0000", 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8241000000000001e-05 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2192 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2572288~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866249+0000", 2024-09-16T11:43:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.4817303629999996, 2024-09-16T11:43:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014184179999999999, 2024-09-16T11:43:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2192 2024-09-16T11:43:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866249+0000", 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866249+0000", 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866249+0000", 2024-09-16T11:43:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866251+0000", 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1119999999999999e-06 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866257+0000", 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9159999999999998e-06 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866262+0000", 2024-09-16T11:43:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8910000000000003e-06 2024-09-16T11:43:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880330+0000", 2024-09-16T11:43:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014068429 2024-09-16T11:43:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880383+0000", 2024-09-16T11:43:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029000000000003e-05 2024-09-16T11:43:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880433+0000", 2024-09-16T11:43:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9446000000000001e-05 2024-09-16T11:43:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2193 4.15 4:ab17817d:::rbd_data.11c17b8dc4e5.0000000000000001:5 [read 2768896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.866266+0000", 2024-09-16T11:43:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.481712516, 2024-09-16T11:43:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.01427954, 2024-09-16T11:43:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2193 2024-09-16T11:43:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866266+0000", 2024-09-16T11:43:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866266+0000", 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866267+0000", 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7300000000000001e-07 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866269+0000", 2024-09-16T11:43:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.73e-06 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866275+0000", 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0229999999999998e-06 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.866278+0000", 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880454+0000", 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014175755999999999 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880498+0000", 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4033999999999999e-05 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.880546+0000", 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7849999999999998e-05 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2670 4.15 4:aecbca69:::rbd_data.11c197cf718c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.995522+0000", 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.352456439, 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024883367999999999, 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2670 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.995522+0000", 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.995522+0000", 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.995524+0000", 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2810000000000001e-06 2024-09-16T11:43:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996603+0000", 2024-09-16T11:43:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010795329999999999 2024-09-16T11:43:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996616+0000", 2024-09-16T11:43:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2850999999999999e-05 2024-09-16T11:43:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996626+0000", 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0362e-05 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996664+0000", 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7078e-05 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.996757+0000", 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3614000000000002e-05 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.001077+0000", 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0043196570000000002 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.020321+0000", 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.019243871999999999 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.020330+0000", 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2369999999999998e-06 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.020369+0000", 2024-09-16T11:43:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9233999999999997e-05 2024-09-16T11:43:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:39.020406+0000", 2024-09-16T11:43:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6649000000000001e-05 2024-09-16T11:43:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.445 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:46.446 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:43:46.446 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:46.444+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:46.548 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:46.547+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:46.586 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:46.586 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:46.586 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:43:46.586 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:46.594 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:43:46.649 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:46.648+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:46.692 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:46.692 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:46.693 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:46.693 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:46.693 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:46.701 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:43:46.784 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:46.783+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:46.844 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:46.844 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:43:46.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:43:46.844 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:43:46.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.844 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(4.18 epoch 34/28 rep_tid 399 entries 34'3 (0'0) error 4:1b51868d:::rbd_data.11c130c94979.0000000000000008:head by client.4545.0:1758 0.000000 -2 [r=-2+0b,r=0+0b] ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 31'1)", 2024-09-16T11:43:46.845 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.858107+0000", 2024-09-16T11:43:46.845 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9828754790000005, 2024-09-16T11:43:46.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0055608790000000003, 2024-09-16T11:43:46.845 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.845 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-16T11:43:46.845 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858107+0000", 2024-09-16T11:43:46.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858107+0000", 2024-09-16T11:43:46.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858107+0000", 2024-09-16T11:43:46.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:43:46.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858112+0000", 2024-09-16T11:43:46.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1630000000000004e-06 2024-09-16T11:43:46.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858123+0000", 2024-09-16T11:43:46.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0519e-05 2024-09-16T11:43:46.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858129+0000", 2024-09-16T11:43:46.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6440000000000003e-06 2024-09-16T11:43:46.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.858258+0000", 2024-09-16T11:43:46.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012866700000000001 2024-09-16T11:43:46.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.863667+0000", 2024-09-16T11:43:46.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054095360000000004 2024-09-16T11:43:46.848 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.849 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.849 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.849 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:2469 4.18 e34/28 4:1b51868d:::rbd_data.11c130c94979.0000000000000008:head v 34'6, mlcod=33'2)", 2024-09-16T11:43:46.849 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.901262+0000", 2024-09-16T11:43:46.849 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9397203860000003, 2024-09-16T11:43:46.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066226829999999999, 2024-09-16T11:43:46.849 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.849 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.849 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901262+0000", 2024-09-16T11:43:46.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901262+0000", 2024-09-16T11:43:46.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901262+0000", 2024-09-16T11:43:46.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0500000000000002e-07 2024-09-16T11:43:46.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901270+0000", 2024-09-16T11:43:46.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2520000000000002e-06 2024-09-16T11:43:46.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901284+0000", 2024-09-16T11:43:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.419e-05 2024-09-16T11:43:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901289+0000", 2024-09-16T11:43:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:43:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901329+0000", 2024-09-16T11:43:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9666999999999998e-05 2024-09-16T11:43:46.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.901347+0000", 2024-09-16T11:43:46.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8201000000000001e-05 2024-09-16T11:43:46.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.907850+0000", 2024-09-16T11:43:46.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0065027949999999996 2024-09-16T11:43:46.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.907884+0000", 2024-09-16T11:43:46.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4261999999999998e-05 2024-09-16T11:43:46.854 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.854 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.854 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.854 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2484 4.5 4:a6f13331:::rbd_data.11c130c94979.0000000000000018:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.855 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.912091+0000", 2024-09-16T11:43:46.855 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9288912260000002, 2024-09-16T11:43:46.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0074078199999999999, 2024-09-16T11:43:46.855 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.855 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.855 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2484 2024-09-16T11:43:46.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.855 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912091+0000", 2024-09-16T11:43:46.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912091+0000", 2024-09-16T11:43:46.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912097+0000", 2024-09-16T11:43:46.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6420000000000001e-06 2024-09-16T11:43:46.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912104+0000", 2024-09-16T11:43:46.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.472e-06 2024-09-16T11:43:46.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912122+0000", 2024-09-16T11:43:46.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7728999999999999e-05 2024-09-16T11:43:46.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912129+0000", 2024-09-16T11:43:46.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1369999999999996e-06 2024-09-16T11:43:46.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912182+0000", 2024-09-16T11:43:46.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2800000000000003e-05 2024-09-16T11:43:46.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912259+0000", 2024-09-16T11:43:46.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7628999999999998e-05 2024-09-16T11:43:46.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-16T11:43:46.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.912446+0000", 2024-09-16T11:43:46.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00018684600000000001 2024-09-16T11:43:46.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.913585+0000", 2024-09-16T11:43:46.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011388030000000001 2024-09-16T11:43:46.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919426+0000", 2024-09-16T11:43:46.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0058407909999999997 2024-09-16T11:43:46.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919474+0000", 2024-09-16T11:43:46.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8044999999999998e-05 2024-09-16T11:43:46.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919499+0000", 2024-09-16T11:43:46.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4926000000000002e-05 2024-09-16T11:43:46.862 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.862 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.862 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.862 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:2501 4.5 4:a3a65b20:::rbd_data.11c130c94979.000000000000008d:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:46.862 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.916515+0000", 2024-09-16T11:43:46.862 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9244670729999998, 2024-09-16T11:43:46.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054241840000000003, 2024-09-16T11:43:46.862 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.862 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.863 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2501 2024-09-16T11:43:46.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.863 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916515+0000", 2024-09-16T11:43:46.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916515+0000", 2024-09-16T11:43:46.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916515+0000", 2024-09-16T11:43:46.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0500000000000004e-07 2024-09-16T11:43:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916518+0000", 2024-09-16T11:43:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4889999999999998e-06 2024-09-16T11:43:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916525+0000", 2024-09-16T11:43:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8759999999999999e-06 2024-09-16T11:43:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916529+0000", 2024-09-16T11:43:46.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3139999999999997e-06 2024-09-16T11:43:46.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916570+0000", 2024-09-16T11:43:46.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1013999999999997e-05 2024-09-16T11:43:46.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916646+0000", 2024-09-16T11:43:46.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6200999999999998e-05 2024-09-16T11:43:46.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-16T11:43:46.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.916740+0000", 2024-09-16T11:43:46.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3535000000000005e-05 2024-09-16T11:43:46.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.918799+0000", 2024-09-16T11:43:46.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020588680000000002 2024-09-16T11:43:46.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.921878+0000", 2024-09-16T11:43:46.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030790359999999998 2024-09-16T11:43:46.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.921917+0000", 2024-09-16T11:43:46.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9684999999999997e-05 2024-09-16T11:43:46.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.921939+0000", 2024-09-16T11:43:46.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1661000000000001e-05 2024-09-16T11:43:46.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.869 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.870 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:2504 4.18 e34/28 4:1cbc85dd:::rbd_data.11c130c94979.00000000000000d9:head v 34'8, mlcod=34'4)", 2024-09-16T11:43:46.870 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:38.918999+0000", 2024-09-16T11:43:46.870 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.9219828290000001, 2024-09-16T11:43:46.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054815239999999998, 2024-09-16T11:43:46.870 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.870 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.870 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.918999+0000", 2024-09-16T11:43:46.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.918999+0000", 2024-09-16T11:43:46.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919000+0000", 2024-09-16T11:43:46.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6199999999999997e-07 2024-09-16T11:43:46.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919005+0000", 2024-09-16T11:43:46.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4090000000000002e-06 2024-09-16T11:43:46.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919017+0000", 2024-09-16T11:43:46.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1960999999999999e-05 2024-09-16T11:43:46.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919024+0000", 2024-09-16T11:43:46.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4370000000000004e-06 2024-09-16T11:43:46.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919073+0000", 2024-09-16T11:43:46.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9638999999999997e-05 2024-09-16T11:43:46.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.919092+0000", 2024-09-16T11:43:46.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8763999999999999e-05 2024-09-16T11:43:46.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.924429+0000", 2024-09-16T11:43:46.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053370099999999997 2024-09-16T11:43:46.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:38.924481+0000", 2024-09-16T11:43:46.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1641999999999998e-05 2024-09-16T11:43:46.875 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.875 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.875 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.876 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:5360 4.1f 4:f871e0e5:::rbd_data.11c18a2497a7.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:46.876 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.001916+0000", 2024-09-16T11:43:46.876 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.8390655090000001, 2024-09-16T11:43:46.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0069369610000000002, 2024-09-16T11:43:46.876 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.876 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.876 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.876 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.876 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.877 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5360 2024-09-16T11:43:46.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.877 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.001916+0000", 2024-09-16T11:43:46.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.001916+0000", 2024-09-16T11:43:46.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.001918+0000", 2024-09-16T11:43:46.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:43:46.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.001926+0000", 2024-09-16T11:43:46.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5279999999999998e-06 2024-09-16T11:43:46.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.001942+0000", 2024-09-16T11:43:46.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5954999999999999e-05 2024-09-16T11:43:46.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.001949+0000", 2024-09-16T11:43:46.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.34e-06 2024-09-16T11:43:46.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.002019+0000", 2024-09-16T11:43:46.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9601e-05 2024-09-16T11:43:46.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.002077+0000", 2024-09-16T11:43:46.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7868000000000002e-05 2024-09-16T11:43:46.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:46.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.003231+0000", 2024-09-16T11:43:46.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011540299999999999 2024-09-16T11:43:46.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.008792+0000", 2024-09-16T11:43:46.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0055614970000000003 2024-09-16T11:43:46.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.008800+0000", 2024-09-16T11:43:46.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8339999999999999e-06 2024-09-16T11:43:46.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.008829+0000", 2024-09-16T11:43:46.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8982000000000001e-05 2024-09-16T11:43:46.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.008853+0000", 2024-09-16T11:43:46.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4592000000000001e-05 2024-09-16T11:43:46.883 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.883 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.883 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.883 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:5665 4.1f 4:fc0e8e36:::rbd_data.11c18a2497a7.000000000000000b:head [truncate 3670016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:46.883 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.361988+0000", 2024-09-16T11:43:46.883 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4789944009999996, 2024-09-16T11:43:46.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062122430000000001, 2024-09-16T11:43:46.884 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.884 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.884 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.884 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.884 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.884 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5665 2024-09-16T11:43:46.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.884 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.361988+0000", 2024-09-16T11:43:46.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.361988+0000", 2024-09-16T11:43:46.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.361988+0000", 2024-09-16T11:43:46.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4600000000000005e-07 2024-09-16T11:43:46.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.361991+0000", 2024-09-16T11:43:46.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5890000000000001e-06 2024-09-16T11:43:46.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.361998+0000", 2024-09-16T11:43:46.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3599999999999998e-06 2024-09-16T11:43:46.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362003+0000", 2024-09-16T11:43:46.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.425e-06 2024-09-16T11:43:46.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362369+0000", 2024-09-16T11:43:46.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00036655599999999998 2024-09-16T11:43:46.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362438+0000", 2024-09-16T11:43:46.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9141999999999996e-05 2024-09-16T11:43:46.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:46.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362768+0000", 2024-09-16T11:43:46.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032945899999999999 2024-09-16T11:43:46.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.367897+0000", 2024-09-16T11:43:46.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051293060000000001 2024-09-16T11:43:46.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.368160+0000", 2024-09-16T11:43:46.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026256899999999999 2024-09-16T11:43:46.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.368181+0000", 2024-09-16T11:43:46.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1297999999999998e-05 2024-09-16T11:43:46.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.368200+0000", 2024-09-16T11:43:46.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8992999999999999e-05 2024-09-16T11:43:46.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.890 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.891 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:5667 4.1f 4:f871e0e5:::rbd_data.11c18a2497a7.000000000000000d:head [truncate 3670016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:46.891 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.362010+0000", 2024-09-16T11:43:46.891 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4789722860000003, 2024-09-16T11:43:46.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062621719999999999, 2024-09-16T11:43:46.891 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.891 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.891 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5667 2024-09-16T11:43:46.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.892 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362010+0000", 2024-09-16T11:43:46.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362010+0000", 2024-09-16T11:43:46.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362010+0000", 2024-09-16T11:43:46.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:43:46.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362012+0000", 2024-09-16T11:43:46.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9829999999999999e-06 2024-09-16T11:43:46.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362018+0000", 2024-09-16T11:43:46.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1569999999999997e-06 2024-09-16T11:43:46.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362022+0000", 2024-09-16T11:43:46.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7230000000000002e-06 2024-09-16T11:43:46.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.362971+0000", 2024-09-16T11:43:46.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00094911799999999999 2024-09-16T11:43:46.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.363056+0000", 2024-09-16T11:43:46.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5334999999999995e-05 2024-09-16T11:43:46.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:46.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.363167+0000", 2024-09-16T11:43:46.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000110214 2024-09-16T11:43:46.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.367906+0000", 2024-09-16T11:43:46.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047389420000000003 2024-09-16T11:43:46.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.368227+0000", 2024-09-16T11:43:46.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032165999999999998 2024-09-16T11:43:46.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.368253+0000", 2024-09-16T11:43:46.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5984000000000001e-05 2024-09-16T11:43:46.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.368272+0000", 2024-09-16T11:43:46.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8620000000000001e-05 2024-09-16T11:43:46.897 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.897 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.898 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.898 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6011 4.1f 4:f871e0e5:::rbd_data.11c18a2497a7.000000000000000d:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:46.898 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.782009+0000", 2024-09-16T11:43:46.898 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.0589730849999999, 2024-09-16T11:43:46.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.006894753, 2024-09-16T11:43:46.898 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.898 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.899 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6011 2024-09-16T11:43:46.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.899 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.782009+0000", 2024-09-16T11:43:46.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.782009+0000", 2024-09-16T11:43:46.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.782009+0000", 2024-09-16T11:43:46.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.89e-07 2024-09-16T11:43:46.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.782011+0000", 2024-09-16T11:43:46.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.747e-06 2024-09-16T11:43:46.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.782016+0000", 2024-09-16T11:43:46.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3020000000000002e-06 2024-09-16T11:43:46.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.782020+0000", 2024-09-16T11:43:46.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4580000000000002e-06 2024-09-16T11:43:46.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.783171+0000", 2024-09-16T11:43:46.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001151551 2024-09-16T11:43:46.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.783260+0000", 2024-09-16T11:43:46.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8172999999999998e-05 2024-09-16T11:43:46.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:46.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.783368+0000", 2024-09-16T11:43:46.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000108356 2024-09-16T11:43:46.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.786458+0000", 2024-09-16T11:43:46.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003089854 2024-09-16T11:43:46.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.788847+0000", 2024-09-16T11:43:46.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023892420000000002 2024-09-16T11:43:46.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.788874+0000", 2024-09-16T11:43:46.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6784999999999999e-05 2024-09-16T11:43:46.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.788904+0000", 2024-09-16T11:43:46.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9896e-05 2024-09-16T11:43:46.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.905 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.905 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6160 4.1f 4:fc0e8e36:::rbd_data.11c18a2497a7.000000000000000b:head [truncate 1310720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:46.905 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.950545+0000", 2024-09-16T11:43:46.905 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.890436491, 2024-09-16T11:43:46.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062640730000000002, 2024-09-16T11:43:46.906 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.906 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.906 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6160 2024-09-16T11:43:46.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.906 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950545+0000", 2024-09-16T11:43:46.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950545+0000", 2024-09-16T11:43:46.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950546+0000", 2024-09-16T11:43:46.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2099999999999997e-07 2024-09-16T11:43:46.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950549+0000", 2024-09-16T11:43:46.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.599e-06 2024-09-16T11:43:46.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950555+0000", 2024-09-16T11:43:46.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1460000000000001e-06 2024-09-16T11:43:46.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950559+0000", 2024-09-16T11:43:46.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4320000000000001e-06 2024-09-16T11:43:46.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.951037+0000", 2024-09-16T11:43:46.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047762199999999997 2024-09-16T11:43:46.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.951105+0000", 2024-09-16T11:43:46.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7830999999999995e-05 2024-09-16T11:43:46.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:46.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.951213+0000", 2024-09-16T11:43:46.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000108262 2024-09-16T11:43:46.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.953849+0000", 2024-09-16T11:43:46.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026364439999999999 2024-09-16T11:43:46.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956751+0000", 2024-09-16T11:43:46.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0029016020000000001 2024-09-16T11:43:46.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956786+0000", 2024-09-16T11:43:46.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4904999999999999e-05 2024-09-16T11:43:46.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956810+0000", 2024-09-16T11:43:46.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3708999999999999e-05 2024-09-16T11:43:46.912 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.912 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.912 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.912 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6162 4.1f 4:f871e0e5:::rbd_data.11c18a2497a7.000000000000000d:head [truncate 1310720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:46.912 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:42.950591+0000", 2024-09-16T11:43:46.913 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.8903912059999999, 2024-09-16T11:43:46.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.006297268, 2024-09-16T11:43:46.913 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.913 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.913 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6162 2024-09-16T11:43:46.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.913 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950591+0000", 2024-09-16T11:43:46.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950591+0000", 2024-09-16T11:43:46.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950592+0000", 2024-09-16T11:43:46.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5900000000000005e-07 2024-09-16T11:43:46.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950596+0000", 2024-09-16T11:43:46.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7350000000000004e-06 2024-09-16T11:43:46.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950609+0000", 2024-09-16T11:43:46.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2656e-05 2024-09-16T11:43:46.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.950616+0000", 2024-09-16T11:43:46.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9319999999999999e-06 2024-09-16T11:43:46.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.951460+0000", 2024-09-16T11:43:46.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00084344300000000004 2024-09-16T11:43:46.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.951535+0000", 2024-09-16T11:43:46.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5768999999999999e-05 2024-09-16T11:43:46.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:46.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.951646+0000", 2024-09-16T11:43:46.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011105 2024-09-16T11:43:46.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956549+0000", 2024-09-16T11:43:46.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049029329999999999 2024-09-16T11:43:46.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956842+0000", 2024-09-16T11:43:46.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029260899999999999 2024-09-16T11:43:46.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956869+0000", 2024-09-16T11:43:46.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7010000000000001e-05 2024-09-16T11:43:46.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:42.956888+0000", 2024-09-16T11:43:46.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9171999999999999e-05 2024-09-16T11:43:46.919 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.919 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.920 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.920 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6219 4.1f 4:fd754f81:::rbd_data.11c18a2497a7.0000000000000006:head [truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:46.920 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:43.056806+0000", 2024-09-16T11:43:46.920 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.7841760550000001, 2024-09-16T11:43:46.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0056997819999999996, 2024-09-16T11:43:46.920 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.920 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.920 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.920 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.921 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.921 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6219 2024-09-16T11:43:46.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.921 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056806+0000", 2024-09-16T11:43:46.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056806+0000", 2024-09-16T11:43:46.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056807+0000", 2024-09-16T11:43:46.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7000000000000003e-07 2024-09-16T11:43:46.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056811+0000", 2024-09-16T11:43:46.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3900000000000003e-06 2024-09-16T11:43:46.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056823+0000", 2024-09-16T11:43:46.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1945e-05 2024-09-16T11:43:46.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056829+0000", 2024-09-16T11:43:46.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8529999999999997e-06 2024-09-16T11:43:46.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056881+0000", 2024-09-16T11:43:46.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1425000000000003e-05 2024-09-16T11:43:46.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056967+0000", 2024-09-16T11:43:46.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6638999999999995e-05 2024-09-16T11:43:46.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:46.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057074+0000", 2024-09-16T11:43:46.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010676800000000001 2024-09-16T11:43:46.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.059622+0000", 2024-09-16T11:43:46.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025480659999999999 2024-09-16T11:43:46.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062457+0000", 2024-09-16T11:43:46.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028349759999999999 2024-09-16T11:43:46.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062484+0000", 2024-09-16T11:43:46.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6950000000000001e-05 2024-09-16T11:43:46.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062506+0000", 2024-09-16T11:43:46.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1800000000000001e-05 2024-09-16T11:43:46.927 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.927 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.927 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.928 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6224 4.1f 4:fc0e8e36:::rbd_data.11c18a2497a7.000000000000000b:head [truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:46.928 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:43.056842+0000", 2024-09-16T11:43:46.928 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.784140448, 2024-09-16T11:43:46.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057434119999999998, 2024-09-16T11:43:46.928 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.928 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.928 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6224 2024-09-16T11:43:46.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.929 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056842+0000", 2024-09-16T11:43:46.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056842+0000", 2024-09-16T11:43:46.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056842+0000", 2024-09-16T11:43:46.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2699999999999999e-07 2024-09-16T11:43:46.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056847+0000", 2024-09-16T11:43:46.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4619999999999998e-06 2024-09-16T11:43:46.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056854+0000", 2024-09-16T11:43:46.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:43:46.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056859+0000", 2024-09-16T11:43:46.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5419999999999996e-06 2024-09-16T11:43:46.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057350+0000", 2024-09-16T11:43:46.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00049094600000000003 2024-09-16T11:43:46.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057407+0000", 2024-09-16T11:43:46.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6375999999999997e-05 2024-09-16T11:43:46.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:46.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057508+0000", 2024-09-16T11:43:46.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101265 2024-09-16T11:43:46.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062398+0000", 2024-09-16T11:43:46.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048902570000000003 2024-09-16T11:43:46.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062538+0000", 2024-09-16T11:43:46.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140016 2024-09-16T11:43:46.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062562+0000", 2024-09-16T11:43:46.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4417999999999999e-05 2024-09-16T11:43:46.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062585+0000", 2024-09-16T11:43:46.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2486000000000001e-05 2024-09-16T11:43:46.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.935 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.935 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6226 4.1f 4:f871e0e5:::rbd_data.11c18a2497a7.000000000000000d:head [truncate 3670016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:46.936 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:43.056935+0000", 2024-09-16T11:43:46.936 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.7840467800000002, 2024-09-16T11:43:46.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057163179999999997, 2024-09-16T11:43:46.936 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.936 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.937 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6226 2024-09-16T11:43:46.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.937 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056935+0000", 2024-09-16T11:43:46.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056935+0000", 2024-09-16T11:43:46.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056936+0000", 2024-09-16T11:43:46.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8299999999999995e-07 2024-09-16T11:43:46.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056941+0000", 2024-09-16T11:43:46.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7759999999999997e-06 2024-09-16T11:43:46.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056954+0000", 2024-09-16T11:43:46.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2858e-05 2024-09-16T11:43:46.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.056960+0000", 2024-09-16T11:43:46.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.959e-06 2024-09-16T11:43:46.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057694+0000", 2024-09-16T11:43:46.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00073463999999999995 2024-09-16T11:43:46.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057767+0000", 2024-09-16T11:43:46.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2349999999999997e-05 2024-09-16T11:43:46.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:46.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.057878+0000", 2024-09-16T11:43:46.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000111365 2024-09-16T11:43:46.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062404+0000", 2024-09-16T11:43:46.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045255590000000002 2024-09-16T11:43:46.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062611+0000", 2024-09-16T11:43:46.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020681399999999999 2024-09-16T11:43:46.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062632+0000", 2024-09-16T11:43:46.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1469999999999999e-05 2024-09-16T11:43:46.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.062652+0000", 2024-09-16T11:43:46.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9544000000000001e-05 2024-09-16T11:43:46.943 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.943 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.943 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.943 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6593 4.5 4:a02fd904:::rbd_data.11c18a2497a7.0000000000000001:head [truncate 2621440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:46.943 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:43.473068+0000", 2024-09-16T11:43:46.943 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.3679141050000001, 2024-09-16T11:43:46.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054355610000000002, 2024-09-16T11:43:46.943 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.943 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.944 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.944 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.944 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.944 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6593 2024-09-16T11:43:46.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.944 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473068+0000", 2024-09-16T11:43:46.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473068+0000", 2024-09-16T11:43:46.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473069+0000", 2024-09-16T11:43:46.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.469e-06 2024-09-16T11:43:46.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473082+0000", 2024-09-16T11:43:46.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2317000000000001e-05 2024-09-16T11:43:46.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473096+0000", 2024-09-16T11:43:46.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4256999999999999e-05 2024-09-16T11:43:46.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473103+0000", 2024-09-16T11:43:46.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.742e-06 2024-09-16T11:43:46.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473157+0000", 2024-09-16T11:43:46.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4559999999999999e-05 2024-09-16T11:43:46.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473236+0000", 2024-09-16T11:43:46.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8416000000000003e-05 2024-09-16T11:43:46.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-16T11:43:46.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473360+0000", 2024-09-16T11:43:46.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000124852 2024-09-16T11:43:46.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.475576+0000", 2024-09-16T11:43:46.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022155909999999998 2024-09-16T11:43:46.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.478434+0000", 2024-09-16T11:43:46.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028574590000000001 2024-09-16T11:43:46.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.478482+0000", 2024-09-16T11:43:46.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8123e-05 2024-09-16T11:43:46.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.478503+0000", 2024-09-16T11:43:46.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1775000000000001e-05 2024-09-16T11:43:46.949 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.949 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.949 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.950 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6598 4.1f 4:fd754f81:::rbd_data.11c18a2497a7.0000000000000006:head [truncate 2621440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:46.950 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:43.473123+0000", 2024-09-16T11:43:46.950 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.3678587929999999, 2024-09-16T11:43:46.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054608260000000002, 2024-09-16T11:43:46.950 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.950 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.950 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.950 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.950 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.950 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6598 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473123+0000", 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473123+0000", 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473124+0000", 2024-09-16T11:43:46.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:43:46.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473126+0000", 2024-09-16T11:43:46.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0470000000000001e-06 2024-09-16T11:43:46.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473132+0000", 2024-09-16T11:43:46.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6309999999999997e-06 2024-09-16T11:43:46.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473137+0000", 2024-09-16T11:43:46.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3839999999999999e-06 2024-09-16T11:43:46.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473185+0000", 2024-09-16T11:43:46.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8040000000000001e-05 2024-09-16T11:43:46.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473266+0000", 2024-09-16T11:43:46.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0926000000000004e-05 2024-09-16T11:43:46.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:46.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473408+0000", 2024-09-16T11:43:46.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014239600000000001 2024-09-16T11:43:46.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.478221+0000", 2024-09-16T11:43:46.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048128850000000003 2024-09-16T11:43:46.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.478527+0000", 2024-09-16T11:43:46.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030549100000000002 2024-09-16T11:43:46.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.478554+0000", 2024-09-16T11:43:46.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7161e-05 2024-09-16T11:43:46.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.478584+0000", 2024-09-16T11:43:46.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0321999999999999e-05 2024-09-16T11:43:46.956 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.956 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.956 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.957 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:6605 4.1f 4:f871e0e5:::rbd_data.11c18a2497a7.000000000000000d:head [truncate 2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:46.957 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:43.473394+0000", 2024-09-16T11:43:46.957 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.367587812, 2024-09-16T11:43:46.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053612219999999997, 2024-09-16T11:43:46.957 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.957 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.957 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6605 2024-09-16T11:43:46.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.958 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473394+0000", 2024-09-16T11:43:46.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473394+0000", 2024-09-16T11:43:46.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473395+0000", 2024-09-16T11:43:46.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6700000000000003e-07 2024-09-16T11:43:46.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473397+0000", 2024-09-16T11:43:46.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3920000000000001e-06 2024-09-16T11:43:46.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473404+0000", 2024-09-16T11:43:46.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5849999999999996e-06 2024-09-16T11:43:46.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.473413+0000", 2024-09-16T11:43:46.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1220000000000001e-06 2024-09-16T11:43:46.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.474035+0000", 2024-09-16T11:43:46.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000622548 2024-09-16T11:43:46.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.474144+0000", 2024-09-16T11:43:46.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000108593 2024-09-16T11:43:46.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:46.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.474260+0000", 2024-09-16T11:43:46.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000116463 2024-09-16T11:43:46.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.478264+0000", 2024-09-16T11:43:46.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.004003258 2024-09-16T11:43:46.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.478691+0000", 2024-09-16T11:43:46.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00042779 2024-09-16T11:43:46.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.478721+0000", 2024-09-16T11:43:46.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9017e-05 2024-09-16T11:43:46.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:43.478755+0000", 2024-09-16T11:43:46.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4887e-05 2024-09-16T11:43:46.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.963 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:9114 4.a 4:549715fa:::rbd_data.11c1dc795e15.000000000000000e:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:46.119530+0000", 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.72145155999999999, 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063669269999999997, 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 9114 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.119530+0000", 2024-09-16T11:43:46.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.119530+0000", 2024-09-16T11:43:46.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.119532+0000", 2024-09-16T11:43:46.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7269999999999999e-06 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.119539+0000", 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3069999999999997e-06 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.119555+0000", 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5716e-05 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.119561+0000", 2024-09-16T11:43:46.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8109999999999999e-06 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.119600+0000", 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9249999999999999e-05 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.119652+0000", 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1847e-05 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.120758+0000", 2024-09-16T11:43:46.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011055699999999999 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.125779+0000", 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050211600000000002 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.125792+0000", 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3485e-05 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.125846+0000", 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3677000000000001e-05 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.125897+0000", 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1377e-05 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:9179 4.5 4:a03f235d:::rbd_data.11c1dc795e15.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:46.201943+0000", 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.63903889300000005, 2024-09-16T11:43:46.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028095163999999999, 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 9179 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201943+0000", 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201943+0000", 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201944+0000", 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.283e-06 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201948+0000", 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0210000000000001e-06 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201960+0000", 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1420000000000001e-05 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201965+0000", 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6570000000000002e-06 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201998+0000", 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2882000000000001e-05 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.202046+0000", 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7274000000000002e-05 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.206571+0000", 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045255080000000001 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.229964+0000", 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023392901000000001 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:46.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.229973+0000", 2024-09-16T11:43:46.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8589999999999994e-06 2024-09-16T11:43:46.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:46.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.230010+0000", 2024-09-16T11:43:46.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7261000000000001e-05 2024-09-16T11:43:46.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.230038+0000", 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8098000000000001e-05 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4545.0:9179 4.5 e40/28 ondisk, result = 0)", 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:46.220937+0000", 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.62004461399999999, 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090286180000000004, 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:46.975 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.220937+0000", 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.220937+0000", 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.220938+0000", 2024-09-16T11:43:46.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1100000000000004e-07 2024-09-16T11:43:46.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:46.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.220942+0000", 2024-09-16T11:43:46.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1099999999999996e-06 2024-09-16T11:43:46.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:46.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.220955+0000", 2024-09-16T11:43:46.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2462e-05 2024-09-16T11:43:46.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.220978+0000", 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2756e-05 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.229950+0000", 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0089728470000000008 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.229959+0000", 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1999999999999994e-06 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.229966+0000", 2024-09-16T11:43:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3320000000000003e-06 2024-09-16T11:43:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:46.979 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:46.980 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:43:46.980 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:43:46.980 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-16T11:43:46.980 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:43:46.980 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:43:46.980 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-16T11:43:46.982 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:46.978+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:46.982 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:43:46.987 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-16T11:43:47.081 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:47.080+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:47.091 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:47.092 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T11:43:47.092 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:47.122 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:47.122 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:47.122 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:43:47.122 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:47.130 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:43:47.182 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:47.180+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:47.219 INFO:teuthology.orchestra.run.smithi070.stdout:1765 write 0x2492a55 thru 0x24a035e (0xd90a bytes) 2024-09-16T11:43:47.228 INFO:teuthology.orchestra.run.smithi070.stdout:1767 write 0x3294884 thru 0x32a0de4 (0xc561 bytes) 2024-09-16T11:43:47.234 INFO:teuthology.orchestra.run.smithi070.stdout:1769 writesame 0x272b31c thru 0x2731179 data_size 0x2f(0x5e5e bytes) 2024-09-16T11:43:47.235 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.240 INFO:teuthology.orchestra.run.smithi070.stdout:1772 trunc from 0xbcf25b7 to 0x7bce5f5 2024-09-16T11:43:47.252 INFO:teuthology.orchestra.run.smithi070.stdout:1773 read 0xd97d2f thru 0xd99089 (0x135b bytes) 2024-09-16T11:43:47.252 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.252 INFO:teuthology.orchestra.run.smithi070.stdout:1776 read 0x95226e thru 0x95a54f (0x82e2 bytes) 2024-09-16T11:43:47.253 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.254 INFO:teuthology.orchestra.run.smithi070.stdout:1779 read 0x3a35114 thru 0x3a39ad2 (0x49bf bytes) 2024-09-16T11:43:47.254 INFO:teuthology.orchestra.run.smithi070.stdout:1782 read 0x4e4b2b5 thru 0x4e4c48b (0x11d7 bytes) 2024-09-16T11:43:47.254 INFO:teuthology.orchestra.run.smithi070.stdout:1783 writesame 0x153413e thru 0x15425a8 data_size 0x5(0xe46b bytes) 2024-09-16T11:43:47.261 INFO:teuthology.orchestra.run.smithi070.stdout:1784 writesame 0x3a9d6be thru 0x3aad247 data_size 0x2(0xfb8a bytes) 2024-09-16T11:43:47.278 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:47.278 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:47.278 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:47.278 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:47.278 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:47.283 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:47.282+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:47.285 INFO:teuthology.orchestra.run.smithi070.stdout:1785 trunc from 0x7bce5f5 to 0x892535 2024-09-16T11:43:47.286 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:43:47.305 INFO:teuthology.orchestra.run.smithi070.stdout:1786 read 0x33678c thru 0x3444cf (0xdd44 bytes) 2024-09-16T11:43:47.305 INFO:teuthology.orchestra.run.smithi070.stdout:1787 read 0x4db8ad thru 0x4ddf00 (0x2654 bytes) 2024-09-16T11:43:47.314 INFO:teuthology.orchestra.run.smithi070.stdout:1788 trunc from 0x892535 to 0x7ce8885 2024-09-16T11:43:47.319 INFO:teuthology.orchestra.run.smithi070.stdout:1789 write 0x1efdef9 thru 0x1efee2d (0xf35 bytes) 2024-09-16T11:43:47.321 INFO:teuthology.orchestra.run.smithi070.stdout:1791 read 0x179a0ab thru 0x17a3292 (0x91e8 bytes) 2024-09-16T11:43:47.333 INFO:teuthology.orchestra.run.smithi070.stdout:1792 writesame 0xce0e7f1 thru 0xce0feb2 data_size 0x3(0x16c2 bytes) 2024-09-16T11:43:47.333 INFO:teuthology.orchestra.run.smithi070.stdout:1795 write 0x231e19f thru 0x23212a5 (0x3107 bytes) 2024-09-16T11:43:47.340 INFO:teuthology.orchestra.run.smithi070.stdout:1797 writesame 0xcedc010 thru 0xcee7a99 data_size 0x3f(0xba8a bytes) 2024-09-16T11:43:47.340 INFO:teuthology.orchestra.run.smithi070.stdout:1798 write 0xc162369 thru 0xc169296 (0x6f2e bytes) 2024-09-16T11:43:47.342 INFO:teuthology.orchestra.run.smithi070.stdout:1799 punch from 0x3c11d3c to 0x3c11f9f, (0x263 bytes) 2024-09-16T11:43:47.344 INFO:teuthology.orchestra.run.smithi070.stdout:1800 trunc from 0xcee7a9a to 0xe216496 2024-09-16T11:43:47.349 INFO:teuthology.orchestra.run.smithi070.stdout:1801 read 0xa03cfb4 thru 0xa049921 (0xc96e bytes) 2024-09-16T11:43:47.351 INFO:teuthology.orchestra.run.smithi070.stdout:1803 trunc from 0xe216496 to 0xc74ed52 2024-09-16T11:43:47.356 INFO:teuthology.orchestra.run.smithi070.stdout:1804 write 0xaf2986d thru 0xaf38d4a (0xf4de bytes) 2024-09-16T11:43:47.359 INFO:teuthology.orchestra.run.smithi070.stdout:1805 read 0x1cba639 thru 0x1cc2985 (0x834d bytes) 2024-09-16T11:43:47.359 INFO:teuthology.orchestra.run.smithi070.stdout:1807 read 0x152e983 thru 0x1534d18 (0x6396 bytes) 2024-09-16T11:43:47.359 INFO:teuthology.orchestra.run.smithi070.stdout:1808 read 0x42fbd66 thru 0x42fd309 (0x15a4 bytes) 2024-09-16T11:43:47.363 INFO:teuthology.orchestra.run.smithi070.stdout:1810 write 0xc7c8ec8 thru 0xc7d7c70 (0xeda9 bytes) 2024-09-16T11:43:47.370 INFO:teuthology.orchestra.run.smithi070.stdout:1811 trunc from 0xc7d7c71 to 0x9e571ca 2024-09-16T11:43:47.379 INFO:teuthology.orchestra.run.smithi070.stdout:1812 write 0x1f5e07f thru 0x1f5f4b8 (0x143a bytes) 2024-09-16T11:43:47.382 INFO:teuthology.orchestra.run.smithi070.stdout:1814 writesame 0x2593294 thru 0x259c857 data_size 0x87(0x95c4 bytes) 2024-09-16T11:43:47.383 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:47.381+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:47.385 INFO:teuthology.orchestra.run.smithi070.stdout:1815 writesame 0x2f3cbc0 thru 0x2f46ba2 data_size 0xb(0x9fe3 bytes) 2024-09-16T11:43:47.391 INFO:teuthology.orchestra.run.smithi070.stdout:1816 punch from 0x555f65a to 0x556de84, (0xe82a bytes) 2024-09-16T11:43:47.391 INFO:teuthology.orchestra.run.smithi070.stdout:1819 read 0x6135ccf thru 0x613ab74 (0x4ea6 bytes) 2024-09-16T11:43:47.392 INFO:teuthology.orchestra.run.smithi070.stdout:1820 read 0x61ef5b8 thru 0x61f42e3 (0x4d2c bytes) 2024-09-16T11:43:47.392 INFO:teuthology.orchestra.run.smithi070.stdout:1821 read 0x64bf9bd thru 0x64c5dfc (0x6440 bytes) 2024-09-16T11:43:47.392 INFO:teuthology.orchestra.run.smithi070.stdout:1822 punch from 0x64ec943 to 0x64ee6d0, (0x1d8d bytes) 2024-09-16T11:43:47.392 INFO:teuthology.orchestra.run.smithi070.stdout:1823 read 0x11adc4f thru 0x11b6979 (0x8d2b bytes) 2024-09-16T11:43:47.392 INFO:teuthology.orchestra.run.smithi070.stdout:1824 punch from 0x29a83a4 to 0x29b4ac4, (0xc720 bytes) 2024-09-16T11:43:47.392 INFO:teuthology.orchestra.run.smithi070.stdout:1827 read 0x7d362be thru 0x7d45cef (0xfa32 bytes) 2024-09-16T11:43:47.392 INFO:teuthology.orchestra.run.smithi070.stdout:1828 punch from 0x834cf09 to 0x834e3c5, (0x14bc bytes) 2024-09-16T11:43:47.392 INFO:teuthology.orchestra.run.smithi070.stdout:1829 punch from 0x3c9d20d to 0x3ca5a8f, (0x8882 bytes) 2024-09-16T11:43:47.392 INFO:teuthology.orchestra.run.smithi070.stdout:1831 read 0x4b1dda6 thru 0x4b25119 (0x7374 bytes) 2024-09-16T11:43:47.392 INFO:teuthology.orchestra.run.smithi070.stdout:1832 punch from 0x89e825b to 0x89eb031, (0x2dd6 bytes) 2024-09-16T11:43:47.392 INFO:teuthology.orchestra.run.smithi070.stdout:1833 write 0x898a92a thru 0x8994fe3 (0xa6ba bytes) 2024-09-16T11:43:47.394 INFO:teuthology.orchestra.run.smithi070.stdout:1835 read 0x6a36c3e thru 0x6a3e827 (0x7bea bytes) 2024-09-16T11:43:47.397 INFO:teuthology.orchestra.run.smithi070.stdout:1837 trunc from 0x9e571ca to 0xc25bb2a 2024-09-16T11:43:47.401 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.401 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.405 INFO:teuthology.orchestra.run.smithi070.stdout:1840 write 0xc3a2eb7 thru 0xc3abb91 (0x8cdb bytes) 2024-09-16T11:43:47.409 INFO:teuthology.orchestra.run.smithi070.stdout:1841 write 0x461d61e thru 0x4621146 (0x3b29 bytes) 2024-09-16T11:43:47.413 INFO:teuthology.orchestra.run.smithi070.stdout:1842 write 0x1794d57 thru 0x179538a (0x634 bytes) 2024-09-16T11:43:47.415 INFO:teuthology.orchestra.run.smithi070.stdout:1843 read 0x763c33b thru 0x763e419 (0x20df bytes) 2024-09-16T11:43:47.428 INFO:teuthology.orchestra.run.smithi070.stdout:1845 trunc from 0xc3abb92 to 0x17289f2 2024-09-16T11:43:47.431 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:47.431 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:43:47.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:43:47.432 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:43:47.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.432 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.432 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.718083+0000", 2024-09-16T11:43:47.432 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.710130206999999, 2024-09-16T11:43:47.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061853209999999997, 2024-09-16T11:43:47.432 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.432 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.433 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22 2024-09-16T11:43:47.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.433 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718083+0000", 2024-09-16T11:43:47.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718083+0000", 2024-09-16T11:43:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718084+0000", 2024-09-16T11:43:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-16T11:43:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718153+0000", 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9362000000000004e-05 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718159+0000", 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3060000000000003e-06 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718164+0000", 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3900000000000003e-06 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719004+0000", 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00084073899999999996 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719050+0000", 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5124999999999999e-05 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719125+0000", 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5914999999999994e-05 2024-09-16T11:43:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720731+0000", 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016052589999999999 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724213+0000", 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034819180000000001 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724245+0000", 2024-09-16T11:43:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2725999999999998e-05 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724268+0000", 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2954000000000001e-05 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.718169+0000", 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.710044662999998, 2024-09-16T11:43:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061430139999999996, 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 23 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718169+0000", 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718169+0000", 2024-09-16T11:43:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718169+0000", 2024-09-16T11:43:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:43:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718227+0000", 2024-09-16T11:43:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7714999999999999e-05 2024-09-16T11:43:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718233+0000", 2024-09-16T11:43:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9320000000000001e-06 2024-09-16T11:43:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718236+0000", 2024-09-16T11:43:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:43:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719452+0000", 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001215673 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719501+0000", 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9283000000000002e-05 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719584+0000", 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3182000000000004e-05 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720759+0000", 2024-09-16T11:43:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011749810000000001 2024-09-16T11:43:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724273+0000", 2024-09-16T11:43:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035134649999999999 2024-09-16T11:43:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724294+0000", 2024-09-16T11:43:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1158999999999999e-05 2024-09-16T11:43:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724312+0000", 2024-09-16T11:43:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740999999999998e-05 2024-09-16T11:43:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.718240+0000", 2024-09-16T11:43:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.709973368, 2024-09-16T11:43:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061176770000000002, 2024-09-16T11:43:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 24 2024-09-16T11:43:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718240+0000", 2024-09-16T11:43:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718240+0000", 2024-09-16T11:43:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718240+0000", 2024-09-16T11:43:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:43:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718296+0000", 2024-09-16T11:43:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5365999999999998e-05 2024-09-16T11:43:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718301+0000", 2024-09-16T11:43:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5219999999999998e-06 2024-09-16T11:43:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718308+0000", 2024-09-16T11:43:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5440000000000004e-06 2024-09-16T11:43:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719993+0000", 2024-09-16T11:43:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001685717 2024-09-16T11:43:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720044+0000", 2024-09-16T11:43:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1007000000000001e-05 2024-09-16T11:43:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720123+0000", 2024-09-16T11:43:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8714999999999994e-05 2024-09-16T11:43:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720918+0000", 2024-09-16T11:43:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00079541099999999999 2024-09-16T11:43:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724319+0000", 2024-09-16T11:43:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034004740000000001 2024-09-16T11:43:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724340+0000", 2024-09-16T11:43:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1131000000000001e-05 2024-09-16T11:43:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724358+0000", 2024-09-16T11:43:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7458000000000001e-05 2024-09-16T11:43:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.718312+0000", 2024-09-16T11:43:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.709901653999999, 2024-09-16T11:43:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060983740000000002, 2024-09-16T11:43:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-16T11:43:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718312+0000", 2024-09-16T11:43:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718312+0000", 2024-09-16T11:43:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718312+0000", 2024-09-16T11:43:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:43:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718314+0000", 2024-09-16T11:43:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:43:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718318+0000", 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3970000000000004e-06 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718321+0000", 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4219999999999999e-06 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720425+0000", 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021042690000000002 2024-09-16T11:43:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720474+0000", 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8449000000000003e-05 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720548+0000", 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4276999999999994e-05 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.721085+0000", 2024-09-16T11:43:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000537252 2024-09-16T11:43:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724361+0000", 2024-09-16T11:43:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0032759289999999999 2024-09-16T11:43:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724393+0000", 2024-09-16T11:43:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1930000000000001e-05 2024-09-16T11:43:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724410+0000", 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7002000000000001e-05 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.508189502999997, 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:43:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:43:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:43:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:43:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:43:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:43:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:43:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:43:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:43:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:43:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:43:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:43:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:43:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:43:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:43:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:43:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:43:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:43:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:43:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:43:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:43:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:43:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.508143810999997, 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:43:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:43:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:43:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:43:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:43:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:43:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:43:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:43:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:43:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:43:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:43:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:43:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:43:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:43:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:43:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:43:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:43:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:43:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:43:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:43:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.508086880999997, 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:43:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:43:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:43:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:43:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:43:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:43:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:43:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:43:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:43:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:43:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:43:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:43:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:43:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:43:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:43:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:43:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:43:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:43:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:43:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:43:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.508061513000001, 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:43:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:43:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:43:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:43:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.508009545, 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:43:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:43:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:43:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:43:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:43:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.507928434999997, 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:43:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:43:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:43:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:43:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:43:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:43:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:43:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:43:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:43:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:43:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:43:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:43:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:43:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:43:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:43:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:43:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.507902457, 2024-09-16T11:43:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:43:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:43:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:43:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:43:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:43:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:43:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:43:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:43:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:43:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337703+0000", 2024-09-16T11:43:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.090509789999999, 2024-09-16T11:43:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053663890000000001, 2024-09-16T11:43:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 70 2024-09-16T11:43:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337703+0000", 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337703+0000", 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337704+0000", 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337706+0000", 2024-09-16T11:43:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:43:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337711+0000", 2024-09-16T11:43:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2900000000000002e-06 2024-09-16T11:43:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337714+0000", 2024-09-16T11:43:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3970000000000001e-06 2024-09-16T11:43:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341010+0000", 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0032962220000000001 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341092+0000", 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1996999999999994e-05 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341220+0000", 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012852600000000001 2024-09-16T11:43:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.342994+0000", 2024-09-16T11:43:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001773279 2024-09-16T11:43:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343021+0000", 2024-09-16T11:43:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.758e-05 2024-09-16T11:43:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343049+0000", 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8118000000000001e-05 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343070+0000", 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0330999999999999e-05 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337716+0000", 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.090497040999999, 2024-09-16T11:43:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054005590000000001, 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 71 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337716+0000", 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337716+0000", 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337716+0000", 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337718+0000", 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.657e-06 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337723+0000", 2024-09-16T11:43:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3329999999999996e-06 2024-09-16T11:43:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337726+0000", 2024-09-16T11:43:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.039e-06 2024-09-16T11:43:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341844+0000", 2024-09-16T11:43:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041171699999999999 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341901+0000", 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7717999999999999e-05 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341974+0000", 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2538000000000002e-05 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343017+0000", 2024-09-16T11:43:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010429510000000001 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343079+0000", 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2045000000000007e-05 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343100+0000", 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1154000000000001e-05 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343117+0000", 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6741000000000001e-05 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337730+0000", 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.090483098, 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063357409999999998, 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-16T11:43:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337730+0000", 2024-09-16T11:43:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337730+0000", 2024-09-16T11:43:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337730+0000", 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337785+0000", 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4673000000000003e-05 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337794+0000", 2024-09-16T11:43:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8829999999999995e-06 2024-09-16T11:43:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337798+0000", 2024-09-16T11:43:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1069999999999998e-06 2024-09-16T11:43:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.342223+0000", 2024-09-16T11:43:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.004425216 2024-09-16T11:43:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.342309+0000", 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5424000000000001e-05 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.342451+0000", 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014200899999999999 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343980+0000", 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001529819 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344021+0000", 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0175000000000001e-05 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344046+0000", 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5677000000000002e-05 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344066+0000", 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9551999999999998e-05 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337806+0000", 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.090407223, 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.006307857, 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-16T11:43:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337806+0000", 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337806+0000", 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337807+0000", 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7199999999999999e-07 2024-09-16T11:43:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337812+0000", 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7259999999999999e-06 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337820+0000", 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9079999999999995e-06 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337824+0000", 2024-09-16T11:43:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0740000000000003e-06 2024-09-16T11:43:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343154+0000", 2024-09-16T11:43:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053298640000000001 2024-09-16T11:43:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343203+0000", 2024-09-16T11:43:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8664e-05 2024-09-16T11:43:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343284+0000", 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1211000000000006e-05 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344015+0000", 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00073082900000000001 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344074+0000", 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9416000000000002e-05 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344095+0000", 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0837999999999999e-05 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344114+0000", 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8754999999999999e-05 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:43:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.090383832000001, 2024-09-16T11:43:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:43:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:43:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:43:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:43:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:43:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:43:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.090361901000001, 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:43:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:43:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:43:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:43:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:43:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:43:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:43:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:43:47.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:43:47.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:43:47.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:43:47.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:43:47.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:43:47.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:43:47.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:43:47.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:43:47.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:43:47.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:43:47.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:43:47.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:43:47.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:43:47.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:43:47.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:43:47.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:43:47.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:43:47.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.534 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.534 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.090285012999999, 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:43:47.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:1165 4.17 e32/28 4:ebaef169:::rbd_data.11c17b8dc4e5.0000000000000006:head v 32'6, mlcod=32'4)", 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.382439+0000", 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.045774099000001, 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059190919999999999, 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.540 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382439+0000", 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382439+0000", 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382440+0000", 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382445+0000", 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9280000000000001e-06 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382454+0000", 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2040000000000003e-06 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382459+0000", 2024-09-16T11:43:47.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382510+0000", 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1066000000000002e-05 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382530+0000", 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9673999999999998e-05 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.388328+0000", 2024-09-16T11:43:47.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057977920000000004 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.388358+0000", 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0586999999999997e-05 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:1167 4.11 e32/28 4:8a86a6ca:::rbd_data.11c17b8dc4e5.000000000000000a:head v 32'21, mlcod=32'19)", 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.382770+0000", 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.045443432000001, 2024-09-16T11:43:47.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0056292640000000001, 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382770+0000", 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382770+0000", 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382770+0000", 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382776+0000", 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3009999999999997e-06 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382788+0000", 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2520999999999999e-05 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382795+0000", 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9489999999999999e-06 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382819+0000", 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3649999999999999e-05 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382832+0000", 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3519999999999999e-05 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:47.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.388364+0000", 2024-09-16T11:43:47.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0055314359999999998 2024-09-16T11:43:47.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:47.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:47.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:47.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.388399+0000", 2024-09-16T11:43:47.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5188999999999998e-05 2024-09-16T11:43:47.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.548 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:47.548 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:47.548 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:47.550 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:47.548+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:47.551 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:43:47.553 INFO:teuthology.orchestra.run.smithi070.stdout:1846 write 0x887203 thru 0x894154 (0xcf52 bytes) 2024-09-16T11:43:47.553 INFO:teuthology.orchestra.run.smithi070.stdout:1848 write 0x54a7739 thru 0x54b1c9c (0xa564 bytes) 2024-09-16T11:43:47.553 INFO:teuthology.orchestra.run.smithi070.stdout:1849 writesame 0x776f9fe thru 0x777dd67 data_size 0x5d(0xe36a bytes) 2024-09-16T11:43:47.553 INFO:teuthology.orchestra.run.smithi070.stdout:1850 write 0x632f0fe thru 0x6332e18 (0x3d1b bytes) 2024-09-16T11:43:47.553 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.553 INFO:teuthology.orchestra.run.smithi070.stdout:1852 punch from 0x1e84de2 to 0x1e8537a, (0x598 bytes) 2024-09-16T11:43:47.553 INFO:teuthology.orchestra.run.smithi070.stdout:1853 write 0x105571e thru 0x1058f2d (0x3810 bytes) 2024-09-16T11:43:47.553 INFO:teuthology.orchestra.run.smithi070.stdout:1855 write 0xd711105 thru 0xd717d1b (0x6c17 bytes) 2024-09-16T11:43:47.553 INFO:teuthology.orchestra.run.smithi070.stdout:1856 write 0x82e0d9f thru 0x82e83d9 (0x763b bytes) 2024-09-16T11:43:47.553 INFO:teuthology.orchestra.run.smithi070.stdout:1857 writesame 0x357f7ee thru 0x3584171 data_size 0xa(0x4984 bytes) 2024-09-16T11:43:47.553 INFO:teuthology.orchestra.run.smithi070.stdout:1858 read 0x96d2da6 thru 0x96d7a63 (0x4cbe bytes) 2024-09-16T11:43:47.554 INFO:teuthology.orchestra.run.smithi070.stdout:1859 punch from 0x8721921 to 0x872627e, (0x495d bytes) 2024-09-16T11:43:47.554 INFO:teuthology.orchestra.run.smithi070.stdout:1860 write 0x3b42eb8 thru 0x3b509be (0xdb07 bytes) 2024-09-16T11:43:47.554 INFO:teuthology.orchestra.run.smithi070.stdout:1861 read 0x2fb765b thru 0x2fbd0c4 (0x5a6a bytes) 2024-09-16T11:43:47.554 INFO:teuthology.orchestra.run.smithi070.stdout:1863 write 0x4f95bb9 thru 0x4f98470 (0x28b8 bytes) 2024-09-16T11:43:47.554 INFO:teuthology.orchestra.run.smithi070.stdout:1865 read 0x14fbe43 thru 0x14fcf2a (0x10e8 bytes) 2024-09-16T11:43:47.554 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.554 INFO:teuthology.orchestra.run.smithi070.stdout:1867 write 0x944b045 thru 0x94543b0 (0x936c bytes) 2024-09-16T11:43:47.554 INFO:teuthology.orchestra.run.smithi070.stdout:1869 write 0xcf07bdc thru 0xcf13a9a (0xbebf bytes) 2024-09-16T11:43:47.554 INFO:teuthology.orchestra.run.smithi070.stdout:1870 trunc from 0xd717d1c to 0xca63d87 2024-09-16T11:43:47.554 INFO:teuthology.orchestra.run.smithi070.stdout:1871 read 0x4ba2700 thru 0x4bb06ef (0xdff0 bytes) 2024-09-16T11:43:47.554 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.554 INFO:teuthology.orchestra.run.smithi070.stdout:1874 write 0xee4b56a thru 0xee4f230 (0x3cc7 bytes) 2024-09-16T11:43:47.554 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.555 INFO:teuthology.orchestra.run.smithi070.stdout:1878 read 0xde6ea6a thru 0xde7b2c9 (0xc860 bytes) 2024-09-16T11:43:47.555 INFO:teuthology.orchestra.run.smithi070.stdout:1879 writesame 0x1507e7b thru 0x150e661 data_size 0x1(0x67e7 bytes) 2024-09-16T11:43:47.555 INFO:teuthology.orchestra.run.smithi070.stdout:1880 trunc from 0xee4f231 to 0x4c9084f 2024-09-16T11:43:47.559 INFO:teuthology.orchestra.run.smithi070.stdout:1881 write 0x3c53148 thru 0x3c59eac (0x6d65 bytes) 2024-09-16T11:43:47.562 INFO:teuthology.orchestra.run.smithi070.stdout:1884 read 0x1e08e92 thru 0x1e0bd7d (0x2eec bytes) 2024-09-16T11:43:47.562 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.562 INFO:teuthology.orchestra.run.smithi070.stdout:1887 write 0x31747ad thru 0x317b141 (0x6995 bytes) 2024-09-16T11:43:47.572 INFO:teuthology.orchestra.run.smithi070.stdout:1888 write 0x647e70f thru 0x6488b98 (0xa48a bytes) 2024-09-16T11:43:47.578 INFO:teuthology.orchestra.run.smithi070.stdout:1890 read 0x31453c8 thru 0x3150a15 (0xb64e bytes) 2024-09-16T11:43:47.583 INFO:teuthology.orchestra.run.smithi070.stdout:1891 trunc from 0x6488b99 to 0x2cadbea 2024-09-16T11:43:47.593 INFO:teuthology.orchestra.run.smithi070.stdout:1893 read 0x2502fef thru 0x25064b6 (0x34c8 bytes) 2024-09-16T11:43:47.606 INFO:teuthology.orchestra.run.smithi070.stdout:1894 trunc from 0x2cadbea to 0xd7c7856 2024-09-16T11:43:47.612 INFO:teuthology.orchestra.run.smithi070.stdout:1895 punch from 0x13018f7 to 0x1311845, (0xff4e bytes) 2024-09-16T11:43:47.612 INFO:teuthology.orchestra.run.smithi070.stdout:1896 read 0x953e7e6 thru 0x954aa3d (0xc258 bytes) 2024-09-16T11:43:47.612 INFO:teuthology.orchestra.run.smithi070.stdout:1898 read 0x2c0ba79 thru 0x2c194a1 (0xda29 bytes) 2024-09-16T11:43:47.612 INFO:teuthology.orchestra.run.smithi070.stdout:1899 write 0x1282f7b thru 0x12889a0 (0x5a26 bytes) 2024-09-16T11:43:47.614 INFO:teuthology.orchestra.run.smithi070.stdout:1900 read 0x5ced7e thru 0x5d98e2 (0xab65 bytes) 2024-09-16T11:43:47.629 INFO:teuthology.orchestra.run.smithi070.stdout:1901 trunc from 0xd7c7856 to 0x1674211 2024-09-16T11:43:47.651 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:47.648+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:47.657 INFO:teuthology.orchestra.run.smithi070.stdout:1902 writesame 0xbbdb8a3 thru 0xbbe7402 data_size 0x10(0xbb60 bytes) 2024-09-16T11:43:47.666 INFO:teuthology.orchestra.run.smithi070.stdout:1903 write 0xe4c2bf8 thru 0xe4c8653 (0x5a5c bytes) 2024-09-16T11:43:47.667 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:47.667 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:47.668 INFO:teuthology.orchestra.run.smithi070.stdout:1905 writesame 0xd739b16 thru 0xd73f1e1 data_size 0x65(0x56cc bytes) 2024-09-16T11:43:47.671 INFO:teuthology.orchestra.run.smithi070.stdout:1906 write 0x5d13177 thru 0x5d21f12 (0xed9c bytes) 2024-09-16T11:43:47.675 INFO:teuthology.orchestra.run.smithi070.stdout:1907 read 0xa078a6c thru 0xa081414 (0x89a9 bytes) 2024-09-16T11:43:47.675 INFO:teuthology.orchestra.run.smithi070.stdout:1908 write 0x5cfe30 thru 0x5d6455 (0x6626 bytes) 2024-09-16T11:43:47.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:43:47.678 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.678 INFO:teuthology.orchestra.run.smithi070.stdout:1910 write 0xdec10e6 thru 0xdecffac (0xeec7 bytes) 2024-09-16T11:43:47.680 INFO:teuthology.orchestra.run.smithi070.stdout:1911 write 0x7402432 thru 0x740df95 (0xbb64 bytes) 2024-09-16T11:43:47.692 INFO:teuthology.orchestra.run.smithi070.stdout:1912 trunc from 0xe4c8654 to 0x60768f6 2024-09-16T11:43:47.702 INFO:teuthology.orchestra.run.smithi070.stdout:1913 read 0x593d023 thru 0x594c737 (0xf715 bytes) 2024-09-16T11:43:47.702 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.702 INFO:teuthology.orchestra.run.smithi070.stdout:1915 read 0x3fc8de7 thru 0x3fd676e (0xd988 bytes) 2024-09-16T11:43:47.702 INFO:teuthology.orchestra.run.smithi070.stdout:1916 read 0x5a79c7e thru 0x5a7e8c5 (0x4c48 bytes) 2024-09-16T11:43:47.702 INFO:teuthology.orchestra.run.smithi070.stdout:1917 read 0x4da2848 thru 0x4da6db0 (0x4569 bytes) 2024-09-16T11:43:47.703 INFO:teuthology.orchestra.run.smithi070.stdout:1918 trunc from 0x60768f6 to 0x5391a44 2024-09-16T11:43:47.725 INFO:teuthology.orchestra.run.smithi070.stdout:1919 write 0xc4c67bd thru 0xc4d39cc (0xd210 bytes) 2024-09-16T11:43:47.728 INFO:teuthology.orchestra.run.smithi070.stdout:1920 write 0x5bb9b6a thru 0x5bc51d9 (0xb670 bytes) 2024-09-16T11:43:47.733 INFO:teuthology.orchestra.run.smithi070.stdout:1921 punch from 0xa87382c to 0xa87af41, (0x7715 bytes) 2024-09-16T11:43:47.734 INFO:teuthology.orchestra.run.smithi070.stdout:1922 read 0x880ff55 thru 0x881aba6 (0xac52 bytes) 2024-09-16T11:43:47.734 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.734 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.734 INFO:teuthology.orchestra.run.smithi070.stdout:1928 writesame 0x47a4574 thru 0x47a4853 data_size 0x8(0x2e0 bytes) 2024-09-16T11:43:47.736 INFO:teuthology.orchestra.run.smithi070.stdout:1929 write 0x26bc6d6 thru 0x26bdd16 (0x1641 bytes) 2024-09-16T11:43:47.738 INFO:teuthology.orchestra.run.smithi070.stdout:1930 read 0xa55370f thru 0xa55bd39 (0x862b bytes) 2024-09-16T11:43:47.739 INFO:teuthology.orchestra.run.smithi070.stdout:1931 write 0xbd2fdd7 thru 0xbd3ba7e (0xbca8 bytes) 2024-09-16T11:43:47.741 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.741 INFO:teuthology.orchestra.run.smithi070.stdout:1934 writesame 0x72e06bc thru 0x72e4829 data_size 0x19(0x416e bytes) 2024-09-16T11:43:47.744 INFO:teuthology.orchestra.run.smithi070.stdout:1935 punch from 0xbf39df4 to 0xbf3e2c6, (0x44d2 bytes) 2024-09-16T11:43:47.744 INFO:teuthology.orchestra.run.smithi070.stdout:1936 read 0xb4d6f43 thru 0xb4e41e5 (0xd2a3 bytes) 2024-09-16T11:43:47.745 INFO:teuthology.orchestra.run.smithi070.stdout:1937 write 0x43f81df thru 0x44078ab (0xf6cd bytes) 2024-09-16T11:43:47.748 INFO:teuthology.orchestra.run.smithi070.stdout:1938 read 0x1e5ec57 thru 0x1e64d3a (0x60e4 bytes) 2024-09-16T11:43:47.749 INFO:teuthology.orchestra.run.smithi070.stdout:1939 write 0x1d42443 thru 0x1d46711 (0x42cf bytes) 2024-09-16T11:43:47.751 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:47.749+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:47.751 INFO:teuthology.orchestra.run.smithi070.stdout:1941 punch from 0x51bb46e to 0x51bb8bb, (0x44d bytes) 2024-09-16T11:43:47.751 INFO:teuthology.orchestra.run.smithi070.stdout:1942 write 0x647b91b thru 0x647be9a (0x580 bytes) 2024-09-16T11:43:47.753 INFO:teuthology.orchestra.run.smithi070.stdout:1943 read 0x3ed5d9a thru 0x3edf730 (0x9997 bytes) 2024-09-16T11:43:47.754 INFO:teuthology.orchestra.run.smithi070.stdout:1944 write 0x5d0601b thru 0x5d1596b (0xf951 bytes) 2024-09-16T11:43:47.757 INFO:teuthology.orchestra.run.smithi070.stdout:1945 punch from 0x54e7135 to 0x54f1fdf, (0xaeaa bytes) 2024-09-16T11:43:47.757 INFO:teuthology.orchestra.run.smithi070.stdout:1946 punch from 0x9e77346 to 0x9e827c9, (0xb483 bytes) 2024-09-16T11:43:47.766 INFO:teuthology.orchestra.run.smithi070.stdout:1949 trunc from 0xc4d39cd to 0x4f0151d 2024-09-16T11:43:47.779 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.779 INFO:teuthology.orchestra.run.smithi070.stdout:1951 write 0x3fbc8cd thru 0x3fbf5aa (0x2cde bytes) 2024-09-16T11:43:47.780 INFO:teuthology.orchestra.run.smithi070.stdout:1952 write 0x146308a thru 0x1466660 (0x35d7 bytes) 2024-09-16T11:43:47.783 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.783 INFO:teuthology.orchestra.run.smithi070.stdout:1955 read 0x212ebb0 thru 0x213bf9a (0xd3eb bytes) 2024-09-16T11:43:47.793 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:47.793 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:47.793 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:47.793 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:47.793 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:47.796 INFO:teuthology.orchestra.run.smithi070.stdout:1956 write 0xb1fbdce thru 0xb2011f4 (0x5427 bytes) 2024-09-16T11:43:47.800 INFO:teuthology.orchestra.run.smithi070.stdout:1957 read 0xc05942 thru 0xc07be5 (0x22a4 bytes) 2024-09-16T11:43:47.800 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.800 INFO:teuthology.orchestra.run.smithi070.stdout:1959 punch from 0x856a00c to 0x856ad40, (0xd34 bytes) 2024-09-16T11:43:47.800 INFO:teuthology.orchestra.run.smithi070.stdout:1960 read 0x61ae814 thru 0x61b1e03 (0x35f0 bytes) 2024-09-16T11:43:47.803 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:43:47.807 INFO:teuthology.orchestra.run.smithi070.stdout:1961 writesame 0xc9cda73 thru 0xc9d37ba data_size 0x28(0x5d48 bytes) 2024-09-16T11:43:47.807 INFO:teuthology.orchestra.run.smithi070.stdout:1963 write 0x400f015 thru 0x401727e (0x826a bytes) 2024-09-16T11:43:47.810 INFO:teuthology.orchestra.run.smithi070.stdout:1964 write 0x1df9768 thru 0x1dfc5f8 (0x2e91 bytes) 2024-09-16T11:43:47.811 INFO:teuthology.orchestra.run.smithi070.stdout:1965 write 0x9e494a0 thru 0x9e4d62c (0x418d bytes) 2024-09-16T11:43:47.814 INFO:teuthology.orchestra.run.smithi070.stdout:1966 writesame 0xad3771f thru 0xad39f82 data_size 0x37(0x2864 bytes) 2024-09-16T11:43:47.817 INFO:teuthology.orchestra.run.smithi070.stdout:1967 read 0x4a36bed thru 0x4a430cc (0xc4e0 bytes) 2024-09-16T11:43:47.827 INFO:teuthology.orchestra.run.smithi070.stdout:1969 trunc from 0xc9d37bb to 0x41a84e4 2024-09-16T11:43:47.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:43:47.852 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:47.850+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:47.852 INFO:teuthology.orchestra.run.smithi070.stdout:1970 writesame 0xc0e8871 thru 0xc0f8396 data_size 0x3e(0xfb26 bytes) 2024-09-16T11:43:47.853 INFO:teuthology.orchestra.run.smithi070.stdout:1971 writesame 0x3d57f70 thru 0x3d5d79a data_size 0x1(0x582b bytes) 2024-09-16T11:43:47.864 INFO:teuthology.orchestra.run.smithi070.stdout:1972 write 0x4bb6816 thru 0x4bb7e8d (0x1678 bytes) 2024-09-16T11:43:47.872 INFO:teuthology.orchestra.run.smithi070.stdout:1973 write 0xc15994b thru 0xc15c15e (0x2814 bytes) 2024-09-16T11:43:47.875 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.875 INFO:teuthology.orchestra.run.smithi070.stdout:1975 writesame 0x45eb949 thru 0x45f79c3 data_size 0x1(0xc07b bytes) 2024-09-16T11:43:47.895 INFO:teuthology.orchestra.run.smithi070.stdout:1976 read 0x284994c thru 0x2850fcc (0x7681 bytes) 2024-09-16T11:43:47.895 INFO:teuthology.orchestra.run.smithi070.stdout:1977 read 0x6149533 thru 0x614fd7f (0x684d bytes) 2024-09-16T11:43:47.895 INFO:teuthology.orchestra.run.smithi070.stdout:1978 writesame 0xb6fcfa0 thru 0xb70b8cf data_size 0x70(0xe930 bytes) 2024-09-16T11:43:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:1979 write 0x95e6f1d thru 0x95e9ba7 (0x2c8b bytes) 2024-09-16T11:43:47.901 INFO:teuthology.orchestra.run.smithi070.stdout:1980 read 0x6ad44f1 thru 0x6ada791 (0x62a1 bytes) 2024-09-16T11:43:47.902 INFO:teuthology.orchestra.run.smithi070.stdout:1981 writesame 0x7d87cd thru 0x7e5c80 data_size 0x2(0xd4b4 bytes) 2024-09-16T11:43:47.914 INFO:teuthology.orchestra.run.smithi070.stdout:1982 read 0x3b69495 thru 0x3b78f41 (0xfaad bytes) 2024-09-16T11:43:47.915 INFO:teuthology.orchestra.run.smithi070.stdout:1983 punch from 0xa710b7d to 0xa720309, (0xf78c bytes) 2024-09-16T11:43:47.915 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:47.915 INFO:teuthology.orchestra.run.smithi070.stdout:1986 write 0x56b28c4 thru 0x56bd8ab (0xafe8 bytes) 2024-09-16T11:43:47.919 INFO:teuthology.orchestra.run.smithi070.stdout:1987 write 0xbf32d3b thru 0xbf3a87c (0x7b42 bytes) 2024-09-16T11:43:47.920 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:47.920 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:47.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:47.921 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:47.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.921 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2367 4.6 4:628f9569:::rbd_data.11c130c94979.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:47.921 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.890791+0000", 2024-09-16T11:43:47.921 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.0262328780000001, 2024-09-16T11:43:47.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0050957820000000001, 2024-09-16T11:43:47.921 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:47.921 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.921 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:47.921 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:47.921 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:47.921 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2367 2024-09-16T11:43:47.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.922 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:47.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:47.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.890791+0000", 2024-09-16T11:43:47.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:47.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.890791+0000", 2024-09-16T11:43:47.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:47.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.890794+0000", 2024-09-16T11:43:47.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3939999999999999e-06 2024-09-16T11:43:47.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:47.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891207+0000", 2024-09-16T11:43:47.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00041353000000000001 2024-09-16T11:43:47.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:47.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891231+0000", 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3918999999999999e-05 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891241+0000", 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9250000000000007e-06 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891294+0000", 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3044999999999998e-05 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:47.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891374+0000", 2024-09-16T11:43:47.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0028e-05 2024-09-16T11:43:47.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:47.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891638+0000", 2024-09-16T11:43:47.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000263486 2024-09-16T11:43:47.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:47.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.895515+0000", 2024-09-16T11:43:47.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0038769339999999998 2024-09-16T11:43:47.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.895782+0000", 2024-09-16T11:43:47.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026755600000000001 2024-09-16T11:43:47.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:47.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.895841+0000", 2024-09-16T11:43:47.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8736999999999997e-05 2024-09-16T11:43:47.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:47.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.895887+0000", 2024-09-16T11:43:47.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6227999999999999e-05 2024-09-16T11:43:47.927 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.927 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:47.927 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.927 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2368 4.6 4:628f9569:::rbd_data.11c130c94979.0000000000000006:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:47.927 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.891258+0000", 2024-09-16T11:43:47.927 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.0257661840000001, 2024-09-16T11:43:47.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.005177755, 2024-09-16T11:43:47.927 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:47.927 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.928 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:47.928 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:47.928 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:47.928 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2368 2024-09-16T11:43:47.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.928 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:47.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:47.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891258+0000", 2024-09-16T11:43:47.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:47.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891258+0000", 2024-09-16T11:43:47.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:47.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891259+0000", 2024-09-16T11:43:47.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2699999999999999e-06 2024-09-16T11:43:47.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:47.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891263+0000", 2024-09-16T11:43:47.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.382e-06 2024-09-16T11:43:47.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891270+0000", 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4050000000000003e-06 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891275+0000", 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4429999999999996e-06 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.892982+0000", 2024-09-16T11:43:47.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001707241 2024-09-16T11:43:47.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:47.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.893050+0000", 2024-09-16T11:43:47.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.779e-05 2024-09-16T11:43:47.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:47.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.893231+0000", 2024-09-16T11:43:47.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0001818 2024-09-16T11:43:47.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.895928+0000", 2024-09-16T11:43:47.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0026962240000000001 2024-09-16T11:43:47.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:47.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.896350+0000", 2024-09-16T11:43:47.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00042250099999999999 2024-09-16T11:43:47.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:47.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.896394+0000", 2024-09-16T11:43:47.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4045999999999998e-05 2024-09-16T11:43:47.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:47.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.896436+0000", 2024-09-16T11:43:47.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1653000000000003e-05 2024-09-16T11:43:47.933 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.933 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:47.933 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.933 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2472 4.6 4:616b0279:::rbd_data.11c130c94979.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:47.933 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.900159+0000", 2024-09-16T11:43:47.933 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.0168655819999994, 2024-09-16T11:43:47.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041559199999999996, 2024-09-16T11:43:47.934 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:47.934 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.934 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:47.934 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:47.934 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:47.934 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2472 2024-09-16T11:43:47.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.934 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:47.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:47.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900159+0000", 2024-09-16T11:43:47.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:47.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900159+0000", 2024-09-16T11:43:47.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:47.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900161+0000", 2024-09-16T11:43:47.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.058e-06 2024-09-16T11:43:47.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:47.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900419+0000", 2024-09-16T11:43:47.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0002588 2024-09-16T11:43:47.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:47.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900440+0000", 2024-09-16T11:43:47.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0786999999999999e-05 2024-09-16T11:43:47.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900449+0000", 2024-09-16T11:43:47.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7520000000000002e-06 2024-09-16T11:43:47.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:47.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900497+0000", 2024-09-16T11:43:47.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7562999999999998e-05 2024-09-16T11:43:47.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:47.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900579+0000", 2024-09-16T11:43:47.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2132000000000006e-05 2024-09-16T11:43:47.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:47.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900806+0000", 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022732699999999999 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.903786+0000", 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029796860000000001 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904243+0000", 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00045775299999999998 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:47.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904281+0000", 2024-09-16T11:43:47.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7323000000000002e-05 2024-09-16T11:43:47.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:47.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904315+0000", 2024-09-16T11:43:47.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3738999999999997e-05 2024-09-16T11:43:47.939 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.939 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:47.939 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.939 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2473 4.6 4:616b0279:::rbd_data.11c130c94979.000000000000000c:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:47.940 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.900942+0000", 2024-09-16T11:43:47.940 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.0160819570000008, 2024-09-16T11:43:47.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037650499999999998, 2024-09-16T11:43:47.940 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:47.940 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.940 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:47.940 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:47.940 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:47.940 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2473 2024-09-16T11:43:47.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.940 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:47.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:47.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900942+0000", 2024-09-16T11:43:47.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:47.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900942+0000", 2024-09-16T11:43:47.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:47.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900943+0000", 2024-09-16T11:43:47.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.099e-06 2024-09-16T11:43:47.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:47.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900949+0000", 2024-09-16T11:43:47.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:43:47.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:47.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900962+0000", 2024-09-16T11:43:47.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.312e-05 2024-09-16T11:43:47.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900969+0000", 2024-09-16T11:43:47.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.323e-06 2024-09-16T11:43:47.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:47.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.902483+0000", 2024-09-16T11:43:47.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015145040000000001 2024-09-16T11:43:47.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:47.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.902575+0000", 2024-09-16T11:43:47.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1372999999999994e-05 2024-09-16T11:43:47.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:47.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.902749+0000", 2024-09-16T11:43:47.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000174105 2024-09-16T11:43:47.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:47.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904325+0000", 2024-09-16T11:43:47.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015760850000000001 2024-09-16T11:43:47.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904625+0000", 2024-09-16T11:43:47.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00030017100000000001 2024-09-16T11:43:47.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:47.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904665+0000", 2024-09-16T11:43:47.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9447999999999999e-05 2024-09-16T11:43:47.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:47.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904707+0000", 2024-09-16T11:43:47.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2645000000000003e-05 2024-09-16T11:43:47.945 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.945 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2784 4.3 4:c63cc7f7:::rbd_data.11c197cf718c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 3940139~50858] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:39.036903+0000", 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 8.8801211149999997, 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003760846, 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2784 2024-09-16T11:43:47.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.947 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:47.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:47.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.036903+0000", 2024-09-16T11:43:47.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:47.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.036903+0000", 2024-09-16T11:43:47.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.036904+0000", 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0610000000000001e-06 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.037359+0000", 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00045501199999999997 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.037372+0000", 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2683999999999999e-05 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.037377+0000", 2024-09-16T11:43:47.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6949999999999999e-06 2024-09-16T11:43:47.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:47.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.037409+0000", 2024-09-16T11:43:47.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2378000000000001e-05 2024-09-16T11:43:47.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:47.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.037499+0000", 2024-09-16T11:43:47.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0000000000000006e-05 2024-09-16T11:43:47.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:47.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.037671+0000", 2024-09-16T11:43:47.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017164900000000001 2024-09-16T11:43:47.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:47.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.039485+0000", 2024-09-16T11:43:47.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018147409999999999 2024-09-16T11:43:47.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.040573+0000", 2024-09-16T11:43:47.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010876320000000001 2024-09-16T11:43:47.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:47.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.040624+0000", 2024-09-16T11:43:47.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1332999999999998e-05 2024-09-16T11:43:47.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:47.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.040664+0000", 2024-09-16T11:43:47.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9660999999999999e-05 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:4961 4.b 4:d70b03a5:::rbd_data.11c18a2497a7.0000000000000026:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2621440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.755037+0000", 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 6.1619869640000005, 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0068672630000000002, 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:47.952 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:47.953 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 4961 2024-09-16T11:43:47.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.953 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:47.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.953 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:47.953 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.755037+0000", 2024-09-16T11:43:47.953 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.953 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:47.953 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.755037+0000", 2024-09-16T11:43:47.953 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.755039+0000", 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8759999999999999e-06 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.756249+0000", 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00121006 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.756267+0000", 2024-09-16T11:43:47.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7785e-05 2024-09-16T11:43:47.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.756276+0000", 2024-09-16T11:43:47.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7460000000000006e-06 2024-09-16T11:43:47.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:47.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.756340+0000", 2024-09-16T11:43:47.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4708000000000005e-05 2024-09-16T11:43:47.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:47.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.756517+0000", 2024-09-16T11:43:47.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017614099999999999 2024-09-16T11:43:47.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:47.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.756805+0000", 2024-09-16T11:43:47.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00028886599999999998 2024-09-16T11:43:47.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:47.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.760673+0000", 2024-09-16T11:43:47.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0038680250000000002 2024-09-16T11:43:47.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.957 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.957 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.761845+0000", 2024-09-16T11:43:47.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011714570000000001 2024-09-16T11:43:47.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.957 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:47.957 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.761879+0000", 2024-09-16T11:43:47.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4233999999999998e-05 2024-09-16T11:43:47.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.957 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:47.958 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.761904+0000", 2024-09-16T11:43:47.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5364999999999999e-05 2024-09-16T11:43:47.958 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.958 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:47.958 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.958 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5357 4.b 4:d70b03a5:::rbd_data.11c18a2497a7.0000000000000026:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:47.958 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.982429+0000", 2024-09-16T11:43:47.958 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 5.934595045, 2024-09-16T11:43:47.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.018203652000000001, 2024-09-16T11:43:47.958 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:47.958 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5357 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982429+0000", 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:47.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982429+0000", 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982431+0000", 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.705e-06 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982438+0000", 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.588e-06 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:47.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982454+0000", 2024-09-16T11:43:47.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5951e-05 2024-09-16T11:43:47.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982460+0000", 2024-09-16T11:43:47.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9329999999999998e-06 2024-09-16T11:43:47.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:47.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982515+0000", 2024-09-16T11:43:47.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4583000000000002e-05 2024-09-16T11:43:47.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982609+0000", 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3857999999999995e-05 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.985720+0000", 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031111020000000001 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000561+0000", 2024-09-16T11:43:47.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014841221999999999 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000571+0000", 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8039999999999996e-06 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000606+0000", 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5197999999999997e-05 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:47.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000633+0000", 2024-09-16T11:43:47.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6707999999999999e-05 2024-09-16T11:43:47.964 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.964 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:47.964 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.964 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.4545.0:5357 4.b e37/28 ondisk, result = 0)", 2024-09-16T11:43:47.964 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.995524+0000", 2024-09-16T11:43:47.964 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 5.9215004899999997, 2024-09-16T11:43:47.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0050393260000000002, 2024-09-16T11:43:47.964 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:47.964 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:47.964 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:47.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:47.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.995524+0000", 2024-09-16T11:43:47.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:47.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.995524+0000", 2024-09-16T11:43:47.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:47.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.995525+0000", 2024-09-16T11:43:47.966 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1489999999999999e-06 2024-09-16T11:43:47.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.966 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:47.966 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.995529+0000", 2024-09-16T11:43:47.966 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.783e-06 2024-09-16T11:43:47.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.966 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:47.966 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.995542+0000", 2024-09-16T11:43:47.966 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3202e-05 2024-09-16T11:43:47.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.995550+0000", 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8369999999999997e-06 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000549+0000", 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0049992769999999999 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000556+0000", 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2860000000000002e-06 2024-09-16T11:43:47.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000563+0000", 2024-09-16T11:43:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7920000000000004e-06 2024-09-16T11:43:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5463 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [stat,truncate 3407872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.143886+0000", 2024-09-16T11:43:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 5.7731384549999998, 2024-09-16T11:43:47.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0052857260000000001, 2024-09-16T11:43:47.969 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:47.969 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.969 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:47.969 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:47.969 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:47.969 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5463 2024-09-16T11:43:47.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.969 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:47.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:47.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143886+0000", 2024-09-16T11:43:47.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:47.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143886+0000", 2024-09-16T11:43:47.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:47.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143888+0000", 2024-09-16T11:43:47.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4889999999999998e-06 2024-09-16T11:43:47.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143901+0000", 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.291e-05 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143924+0000", 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2523000000000001e-05 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143931+0000", 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144015+0000", 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4552e-05 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144111+0000", 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5310999999999996e-05 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144240+0000", 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012950900000000001 2024-09-16T11:43:47.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:47.973 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144947+0000", 2024-09-16T11:43:47.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00070658099999999996 2024-09-16T11:43:47.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.973 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.149102+0000", 2024-09-16T11:43:47.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041549810000000003 2024-09-16T11:43:47.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:47.973 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.149145+0000", 2024-09-16T11:43:47.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3741e-05 2024-09-16T11:43:47.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:47.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.149171+0000", 2024-09-16T11:43:47.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6050999999999998e-05 2024-09-16T11:43:47.974 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.974 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:47.974 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.974 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5488 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [stat,truncate 2621440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:47.974 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.155899+0000", 2024-09-16T11:43:47.975 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 5.761124755, 2024-09-16T11:43:47.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0040559309999999996, 2024-09-16T11:43:47.975 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:47.975 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.975 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:47.975 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:47.975 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:47.975 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5488 2024-09-16T11:43:47.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.975 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:47.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:47.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155899+0000", 2024-09-16T11:43:47.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:47.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155899+0000", 2024-09-16T11:43:47.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:47.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155901+0000", 2024-09-16T11:43:47.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9920000000000002e-06 2024-09-16T11:43:47.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155912+0000", 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0091999999999999e-05 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155933+0000", 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1821999999999999e-05 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155943+0000", 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2560000000000008e-06 2024-09-16T11:43:47.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.156006+0000", 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3286000000000004e-05 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.156106+0000", 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000100181 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.156252+0000", 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000146434 2024-09-16T11:43:47.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.157009+0000", 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000756527 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.159901+0000", 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0028918590000000001 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.159932+0000", 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0642000000000002e-05 2024-09-16T11:43:47.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.159955+0000", 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3839999999999999e-05 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5527 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [stat,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.224549+0000", 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 5.6924750839999998, 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046217860000000001, 2024-09-16T11:43:47.980 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:47.981 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.981 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:47.981 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:47.981 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:47.981 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5527 2024-09-16T11:43:47.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.981 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:47.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:47.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224549+0000", 2024-09-16T11:43:47.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224549+0000", 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224551+0000", 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1100000000000001e-06 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224558+0000", 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2359999999999998e-06 2024-09-16T11:43:47.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224581+0000", 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2506000000000001e-05 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224589+0000", 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8839999999999994e-06 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224652+0000", 2024-09-16T11:43:47.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2791000000000002e-05 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224733+0000", 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1814e-05 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224856+0000", 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0001228 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:47.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.225410+0000", 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00055401500000000004 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.229117+0000", 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037066109999999998 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.229147+0000", 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0238000000000001e-05 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:47.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.229171+0000", 2024-09-16T11:43:47.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3781000000000002e-05 2024-09-16T11:43:47.986 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.986 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:47.986 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.987 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5998 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:47.988 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.781886+0000", 2024-09-16T11:43:47.988 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 5.1351382000000001, 2024-09-16T11:43:47.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041660059999999999, 2024-09-16T11:43:47.988 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:47.988 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.988 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:47.988 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:47.988 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:47.988 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5998 2024-09-16T11:43:47.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.988 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:47.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781886+0000", 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781886+0000", 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781887+0000", 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3099999999999996e-07 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781892+0000", 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8690000000000003e-06 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781905+0000", 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3091999999999999e-05 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781911+0000", 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.251e-06 2024-09-16T11:43:47.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781966+0000", 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4407999999999998e-05 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.782041+0000", 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5638999999999999e-05 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.782162+0000", 2024-09-16T11:43:47.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000120659 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.782663+0000", 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00050085500000000003 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.786004+0000", 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0033415509999999999 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:47.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.786030+0000", 2024-09-16T11:43:47.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5899000000000001e-05 2024-09-16T11:43:47.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:47.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.786052+0000", 2024-09-16T11:43:47.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1852e-05 2024-09-16T11:43:47.993 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.993 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:47.993 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.993 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:6213 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:47.993 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:43.056238+0000", 2024-09-16T11:43:47.994 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8607857140000004, 2024-09-16T11:43:47.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0064010559999999996, 2024-09-16T11:43:47.994 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:47.994 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:47.994 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:47.994 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:47.994 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:47.994 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 6213 2024-09-16T11:43:47.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.994 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:47.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056238+0000", 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056238+0000", 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056241+0000", 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1679999999999998e-06 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:47.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056248+0000", 2024-09-16T11:43:47.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2429999999999999e-06 2024-09-16T11:43:47.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:47.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056267+0000", 2024-09-16T11:43:47.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9511e-05 2024-09-16T11:43:47.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:47.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056278+0000", 2024-09-16T11:43:47.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0915e-05 2024-09-16T11:43:47.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:47.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056336+0000", 2024-09-16T11:43:47.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7349000000000003e-05 2024-09-16T11:43:47.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:47.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056418+0000", 2024-09-16T11:43:47.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2165999999999999e-05 2024-09-16T11:43:47.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056538+0000", 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012048999999999999 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.057059+0000", 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00052033099999999996 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.062554+0000", 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0054957649999999997 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:47.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.062619+0000", 2024-09-16T11:43:47.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4329e-05 2024-09-16T11:43:47.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:47.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:47.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.062640+0000", 2024-09-16T11:43:47.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0789e-05 2024-09-16T11:43:47.999 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.999 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:47.999 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:47.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:47.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:6552 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:43.445001+0000", 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.4720228859999995, 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037872919999999998, 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 6552 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:48.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445001+0000", 2024-09-16T11:43:48.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:48.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445001+0000", 2024-09-16T11:43:48.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:48.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445003+0000", 2024-09-16T11:43:48.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.153e-06 2024-09-16T11:43:48.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:48.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445011+0000", 2024-09-16T11:43:48.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6599999999999995e-06 2024-09-16T11:43:48.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:48.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445031+0000", 2024-09-16T11:43:48.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9939e-05 2024-09-16T11:43:48.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:48.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445040+0000", 2024-09-16T11:43:48.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5259999999999993e-06 2024-09-16T11:43:48.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:48.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445093+0000", 2024-09-16T11:43:48.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3412999999999998e-05 2024-09-16T11:43:48.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:48.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445177+0000", 2024-09-16T11:43:48.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4040000000000002e-05 2024-09-16T11:43:48.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445292+0000", 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000114737 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445965+0000", 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00067304400000000003 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.448739+0000", 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0027745690000000002 2024-09-16T11:43:48.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.448767+0000", 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7990000000000001e-05 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.448789+0000", 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1220999999999999e-05 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:6592 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [truncate 2621440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:43.473057+0000", 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.4439671540000001, 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0054781910000000003, 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 6592 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473057+0000", 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473057+0000", 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473059+0000", 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0660000000000002e-06 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:48.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473072+0000", 2024-09-16T11:43:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2479e-05 2024-09-16T11:43:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473092+0000", 2024-09-16T11:43:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0097000000000001e-05 2024-09-16T11:43:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473107+0000", 2024-09-16T11:43:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5092999999999999e-05 2024-09-16T11:43:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:48.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473155+0000", 2024-09-16T11:43:48.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8532000000000002e-05 2024-09-16T11:43:48.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:48.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473245+0000", 2024-09-16T11:43:48.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9789000000000004e-05 2024-09-16T11:43:48.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:48.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473364+0000", 2024-09-16T11:43:48.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011893200000000001 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473969+0000", 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00060456599999999996 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.478475+0000", 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004506248 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:48.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.478515+0000", 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0219999999999998e-05 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.478535+0000", 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0169e-05 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:9071 4.3 4:c3279cee:::rbd_data.11c1dc795e15.0000000000000014:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.027961+0000", 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.889062821, 2024-09-16T11:43:48.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0078018230000000003, 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 9071 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.027961+0000", 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.027961+0000", 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.027962+0000", 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7899999999999998e-07 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.027966+0000", 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2250000000000002e-06 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.027976+0000", 2024-09-16T11:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9019999999999994e-06 2024-09-16T11:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.027980+0000", 2024-09-16T11:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0330000000000002e-06 2024-09-16T11:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.028016+0000", 2024-09-16T11:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5607e-05 2024-09-16T11:43:48.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:48.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.028060+0000", 2024-09-16T11:43:48.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4502999999999997e-05 2024-09-16T11:43:48.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:48.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.029336+0000", 2024-09-16T11:43:48.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001275782 2024-09-16T11:43:48.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.035687+0000", 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0063505250000000001 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.035710+0000", 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3187999999999999e-05 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.035740+0000", 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0046e-05 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.035763+0000", 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3332999999999998e-05 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:9081 4.b 4:d317bc5b:::rbd_data.11c1dc795e15.000000000000000d:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.8630685819999999, 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027796406999999999, 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:48.017 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:48.018 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:48.018 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:48.018 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:48.018 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 9081 2024-09-16T11:43:48.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.018 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:48.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.018 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:48.018 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:43:48.018 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.018 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:48.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:43:48.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:48.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053957+0000", 2024-09-16T11:43:48.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7010000000000001e-06 2024-09-16T11:43:48.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:48.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053965+0000", 2024-09-16T11:43:48.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5939999999999996e-06 2024-09-16T11:43:48.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053980+0000", 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5099000000000001e-05 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053986+0000", 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5759999999999996e-06 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.054023+0000", 2024-09-16T11:43:48.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999999e-05 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.054076+0000", 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2852999999999997e-05 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.058880+0000", 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048042639999999999 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:48.021 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081663+0000", 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022782908000000001 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081672+0000", 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0440000000000001e-06 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081707+0000", 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4986e-05 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:48.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081752+0000", 2024-09-16T11:43:48.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4781999999999999e-05 2024-09-16T11:43:48.023 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.023 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:48.023 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.023 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.4545.0:9081 4.b e40/28 ondisk, result = 0)", 2024-09-16T11:43:48.023 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.073300+0000", 2024-09-16T11:43:48.023 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.84372424, 2024-09-16T11:43:48.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0083654630000000001, 2024-09-16T11:43:48.023 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:48.023 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:48.024 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:48.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:48.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073300+0000", 2024-09-16T11:43:48.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:48.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073300+0000", 2024-09-16T11:43:48.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073301+0000", 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2100000000000001e-06 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073307+0000", 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.071e-06 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073320+0000", 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3149000000000001e-05 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:48.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073329+0000", 2024-09-16T11:43:48.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4069999999999993e-06 2024-09-16T11:43:48.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:48.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081650+0000", 2024-09-16T11:43:48.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.008321541 2024-09-16T11:43:48.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:48.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081658+0000", 2024-09-16T11:43:48.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6260000000000003e-06 2024-09-16T11:43:48.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:48.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081665+0000", 2024-09-16T11:43:48.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4590000000000001e-06 2024-09-16T11:43:48.027 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.027 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:48.027 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.027 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:9113 4.b 4:d317bc5b:::rbd_data.11c1dc795e15.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:43:48.027 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.119551+0000", 2024-09-16T11:43:48.027 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.7974731230000001, 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.020116319000000001, 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 9113 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119551+0000", 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:48.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119551+0000", 2024-09-16T11:43:48.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:48.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119553+0000", 2024-09-16T11:43:48.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.782e-06 2024-09-16T11:43:48.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:48.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119563+0000", 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0325e-05 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119577+0000", 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.382e-05 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119583+0000", 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.57e-06 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:48.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119620+0000", 2024-09-16T11:43:48.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7737000000000001e-05 2024-09-16T11:43:48.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:48.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119673+0000", 2024-09-16T11:43:48.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3152000000000002e-05 2024-09-16T11:43:48.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:48.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.122945+0000", 2024-09-16T11:43:48.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0032719390000000002 2024-09-16T11:43:48.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139568+0000", 2024-09-16T11:43:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016622839 2024-09-16T11:43:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139606+0000", 2024-09-16T11:43:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8006000000000002e-05 2024-09-16T11:43:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139641+0000", 2024-09-16T11:43:48.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5076000000000001e-05 2024-09-16T11:43:48.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.033 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:48.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139667+0000", 2024-09-16T11:43:48.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6072999999999999e-05 2024-09-16T11:43:48.033 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.033 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:48.033 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.033 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.4545.0:9113 4.b e40/28 ondisk, result = 0)", 2024-09-16T11:43:48.033 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.133655+0000", 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.78336896, 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0060177909999999998, 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.133655+0000", 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.133655+0000", 2024-09-16T11:43:48.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:48.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:48.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.133656+0000", 2024-09-16T11:43:48.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.37e-07 2024-09-16T11:43:48.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:48.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.133660+0000", 2024-09-16T11:43:48.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5700000000000001e-06 2024-09-16T11:43:48.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:48.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.133672+0000", 2024-09-16T11:43:48.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2201000000000001e-05 2024-09-16T11:43:48.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:48.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.133677+0000", 2024-09-16T11:43:48.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3210000000000004e-06 2024-09-16T11:43:48.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:48.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139595+0000", 2024-09-16T11:43:48.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0059179929999999999 2024-09-16T11:43:48.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:48.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139602+0000", 2024-09-16T11:43:48.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0670000000000003e-06 2024-09-16T11:43:48.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:48.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:48.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:48.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139673+0000", 2024-09-16T11:43:48.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0902000000000007e-05 2024-09-16T11:43:48.037 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.037 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:48.037 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.037 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:48.038 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:48.038 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:48.039 INFO:teuthology.orchestra.run.smithi070.stdout:1988 read 0xa528270 thru 0xa538198 (0xff29 bytes) 2024-09-16T11:43:48.039 INFO:teuthology.orchestra.run.smithi070.stdout:1989 read 0xbc3b019 thru 0xbc406e8 (0x56d0 bytes) 2024-09-16T11:43:48.039 INFO:teuthology.orchestra.run.smithi070.stdout:1990 write 0x781b593 thru 0x781eff0 (0x3a5e bytes) 2024-09-16T11:43:48.039 INFO:teuthology.orchestra.run.smithi070.stdout:1991 read 0x30bd934 thru 0x30c933c (0xba09 bytes) 2024-09-16T11:43:48.039 INFO:teuthology.orchestra.run.smithi070.stdout:1992 read 0x3c7d2fe thru 0x3c80b2a (0x382d bytes) 2024-09-16T11:43:48.040 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:43:48.042 INFO:teuthology.orchestra.run.smithi070.stdout:1993 trunc from 0xc15c15f to 0x52da198 2024-09-16T11:43:48.042 INFO:teuthology.orchestra.run.smithi070.stdout:1994 writesame 0x8d4a12d thru 0x8d55c4b data_size 0x1(0xbb1f bytes) 2024-09-16T11:43:48.042 INFO:teuthology.orchestra.run.smithi070.stdout:1995 read 0x3924a5d thru 0x3925863 (0xe07 bytes) 2024-09-16T11:43:48.042 INFO:teuthology.orchestra.run.smithi070.stdout:1996 read 0x4eaa510 thru 0x4eaa7c8 (0x2b9 bytes) 2024-09-16T11:43:48.042 INFO:teuthology.orchestra.run.smithi070.stdout:1997 trunc from 0x8d55c4c to 0x495511f 2024-09-16T11:43:48.042 INFO:teuthology.orchestra.run.smithi070.stdout:1998 trunc from 0x495511f to 0x1fefa10 2024-09-16T11:43:48.042 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:48.042 INFO:teuthology.orchestra.run.smithi070.stdout:2002 trunc from 0x1fefa10 to 0x67849fa 2024-09-16T11:43:48.042 INFO:teuthology.orchestra.run.smithi070.stdout:2003 trunc from 0x67849fa to 0xea199f4 2024-09-16T11:43:48.042 INFO:teuthology.orchestra.run.smithi070.stdout:2004 punch from 0x1c11702 to 0x1c20b23, (0xf421 bytes) 2024-09-16T11:43:48.042 INFO:teuthology.orchestra.run.smithi070.stdout:2005 write 0x5ea8379 thru 0x5eabadc (0x3764 bytes) 2024-09-16T11:43:48.042 INFO:teuthology.orchestra.run.smithi070.stdout:2007 trunc from 0xea199f4 to 0x8f7fded 2024-09-16T11:43:48.042 INFO:teuthology.orchestra.run.smithi070.stdout:2008 write 0xc28ef09 thru 0xc29a8a6 (0xb99e bytes) 2024-09-16T11:43:48.043 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:48.037+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:48.049 INFO:teuthology.orchestra.run.smithi070.stdout:2009 trunc from 0xc29a8a7 to 0x7992d7 2024-09-16T11:43:48.059 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:43:48.062 INFO:teuthology.orchestra.run.smithi070.stdout:2010 punch from 0xa043f to 0xa1ae0, (0x16a1 bytes) 2024-09-16T11:43:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:2011 write 0xc218819 thru 0xc21d78b (0x4f73 bytes) 2024-09-16T11:43:48.088 INFO:teuthology.orchestra.run.smithi070.stdout:2012 trunc from 0xc21d78c to 0xeb23534 2024-09-16T11:43:48.100 INFO:teuthology.orchestra.run.smithi070.stdout:2013 trunc from 0xeb23534 to 0xa15fd7f 2024-09-16T11:43:48.110 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:48.117 INFO:teuthology.orchestra.run.smithi070.stdout:2015 write 0xba7f0a3 thru 0xba8766b (0x85c9 bytes) 2024-09-16T11:43:48.121 INFO:teuthology.orchestra.run.smithi070.stdout:2016 write 0x46faa54 thru 0x4704c35 (0xa1e2 bytes) 2024-09-16T11:43:48.136 INFO:teuthology.orchestra.run.smithi070.stdout:2017 trunc from 0xba8766c to 0x32c33d9 2024-09-16T11:43:48.140 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:48.138+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:48.164 INFO:teuthology.orchestra.run.smithi070.stdout:2018 writesame 0xc375dba thru 0xc37ae44 data_size 0x1d(0x508b bytes) 2024-09-16T11:43:48.172 INFO:teuthology.orchestra.run.smithi070.stdout:2019 write 0xdf3929e thru 0xdf3c52b (0x328e bytes) 2024-09-16T11:43:48.172 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:48.172 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:48.172 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:48.172 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:48.175 INFO:teuthology.orchestra.run.smithi070.stdout:2020 read 0x863099 thru 0x868d60 (0x5cc8 bytes) 2024-09-16T11:43:48.181 INFO:teuthology.orchestra.run.smithi070.stdout:2021 trunc from 0xdf3c52c to 0xa581ebc 2024-09-16T11:43:48.183 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:43:48.201 INFO:teuthology.orchestra.run.smithi070.stdout:2022 write 0xece0f7f thru 0xececa11 (0xba93 bytes) 2024-09-16T11:43:48.204 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:48.210 INFO:teuthology.orchestra.run.smithi070.stdout:2027 trunc from 0xececa12 to 0xa5ad198 2024-09-16T11:43:48.220 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:43:48.220 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:43:48.226 INFO:teuthology.orchestra.run.smithi070.stdout:2028 write 0xb13f72c thru 0xb13ffa6 (0x87b bytes) 2024-09-16T11:43:48.231 INFO:teuthology.orchestra.run.smithi070.stdout:2032 trunc from 0xb13ffa7 to 0xcccbf77 2024-09-16T11:43:48.238 INFO:teuthology.orchestra.run.smithi070.stdout:2036 punch from 0x57295c8 to 0x572e857, (0x528f bytes) 2024-09-16T11:43:48.238 INFO:teuthology.orchestra.run.smithi070.stdout:2037 write 0xbe20ef2 thru 0xbe269f9 (0x5b08 bytes) 2024-09-16T11:43:48.239 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:43:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:43:48.240 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:48.239+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:48.242 INFO:teuthology.orchestra.run.smithi070.stdout:2038 write 0x485b255 thru 0x48660f1 (0xae9d bytes) 2024-09-16T11:43:48.245 INFO:teuthology.orchestra.run.smithi070.stdout:2039 read 0x63c8250 thru 0x63d293d (0xa6ee bytes) 2024-09-16T11:43:48.245 INFO:teuthology.orchestra.run.smithi070.stdout:2041 write 0x5da9164 thru 0x5dad5cd (0x446a bytes) 2024-09-16T11:43:48.248 INFO:teuthology.orchestra.run.smithi070.stdout:2042 punch from 0x252c60e to 0x2539f77, (0xd969 bytes) 2024-09-16T11:43:48.248 INFO:teuthology.orchestra.run.smithi070.stdout:2043 writesame 0x81e3ca3 thru 0x81f0d88 data_size 0x9(0xd0e6 bytes) 2024-09-16T11:43:48.253 INFO:teuthology.orchestra.run.smithi070.stdout:2044 write 0x1ac93d5 thru 0x1acc18e (0x2dba bytes) 2024-09-16T11:43:48.255 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:43:48.256 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:43:48.257 INFO:teuthology.orchestra.run.smithi070.stdout:2045 read 0xb646974 thru 0xb6500f3 (0x9780 bytes) 2024-09-16T11:43:48.257 INFO:teuthology.orchestra.run.smithi070.stdout:2046 read 0xb4cf78e thru 0xb4d3468 (0x3cdb bytes) 2024-09-16T11:43:48.258 INFO:teuthology.orchestra.run.smithi070.stdout:2047 punch from 0x4dd1839 to 0x4dd81d6, (0x699d bytes) 2024-09-16T11:43:48.258 INFO:teuthology.orchestra.run.smithi070.stdout:2048 write 0x75faeb thru 0x7674af (0x79c5 bytes) 2024-09-16T11:43:48.259 INFO:teuthology.orchestra.run.smithi070.stdout:2049 write 0x1c0d1c4 thru 0x1c18ea6 (0xbce3 bytes) 2024-09-16T11:43:48.262 INFO:teuthology.orchestra.run.smithi070.stdout:2050 writesame 0xb8d4fa6 thru 0xb8d775e data_size 0x1(0x27b9 bytes) 2024-09-16T11:43:48.268 INFO:teuthology.orchestra.run.smithi070.stdout:2051 read 0xa6b7a99 thru 0xa6bd809 (0x5d71 bytes) 2024-09-16T11:43:48.269 INFO:teuthology.orchestra.run.smithi070.stdout:2052 write 0x56fcf53 thru 0x570b53c (0xe5ea bytes) 2024-09-16T11:43:48.273 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:43:48.273 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:43:48.275 INFO:teuthology.orchestra.run.smithi070.stdout:2053 write 0x6d53f52 thru 0x6d54c11 (0xcc0 bytes) 2024-09-16T11:43:48.279 INFO:teuthology.orchestra.run.smithi070.stdout:2054 read 0xacb07bb thru 0xacbbacf (0xb315 bytes) 2024-09-16T11:43:48.279 INFO:teuthology.orchestra.run.smithi070.stdout:2055 write 0x600677 thru 0x608190 (0x7b1a bytes) 2024-09-16T11:43:48.280 INFO:teuthology.orchestra.run.smithi070.stdout:2056 punch from 0x5f4dcad to 0x5f54bac, (0x6eff bytes) 2024-09-16T11:43:48.289 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:43:48.289 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:43:48.293 INFO:teuthology.orchestra.run.smithi070.stdout:2058 trunc from 0xcccbf77 to 0x297e848 2024-09-16T11:43:48.303 INFO:teuthology.orchestra.run.smithi070.stdout:2059 read 0xf7c7e5 thru 0xf88698 (0xbeb4 bytes) 2024-09-16T11:43:48.304 INFO:teuthology.orchestra.run.smithi070.stdout:2060 read 0x1941b99 thru 0x194304e (0x14b6 bytes) 2024-09-16T11:43:48.304 INFO:teuthology.orchestra.run.smithi070.stdout:2061 punch from 0xcd25f3 to 0xcdbbf2, (0x95ff bytes) 2024-09-16T11:43:48.306 INFO:teuthology.orchestra.run.smithi070.stdout:2062 trunc from 0x297e848 to 0x43e5d7b 2024-09-16T11:43:48.307 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:43:48.307 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:43:48.315 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:48.316 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:48.322 INFO:teuthology.orchestra.run.smithi070.stdout:2065 trunc from 0x43e5d7b to 0xc829e18 2024-09-16T11:43:48.326 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:43:48.326 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:43:48.329 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:43:48.334 INFO:teuthology.orchestra.run.smithi070.stdout:2066 write 0xe5c0380 thru 0xe5c523e (0x4ebf bytes) 2024-09-16T11:43:48.338 INFO:teuthology.orchestra.run.smithi070.stdout:2067 read 0x2fe6375 thru 0x2fea5f6 (0x4282 bytes) 2024-09-16T11:43:48.338 INFO:teuthology.orchestra.run.smithi070.stdout:2069 punch from 0x108310e to 0x1090126, (0xd018 bytes) 2024-09-16T11:43:48.338 INFO:teuthology.orchestra.run.smithi070.stdout:2071 read 0xcb97a42 thru 0xcb991ff (0x17be bytes) 2024-09-16T11:43:48.338 INFO:teuthology.orchestra.run.smithi070.stdout:2072 read 0x9b1d034 thru 0x9b259a0 (0x896d bytes) 2024-09-16T11:43:48.338 INFO:teuthology.orchestra.run.smithi070.stdout:2074 writesame 0xba90f6d thru 0xba91686 data_size 0x9(0x71a bytes) 2024-09-16T11:43:48.340 INFO:teuthology.orchestra.run.smithi070.stdout:2075 read 0x78043ed thru 0x7805186 (0xd9a bytes) 2024-09-16T11:43:48.341 INFO:teuthology.orchestra.run.smithi070.stdout:2077 writesame 0xed5846 thru 0xede541 data_size 0xe(0x8cfc bytes) 2024-09-16T11:43:48.341 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:48.339+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:48.343 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:43:48.343 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:43:48.357 INFO:teuthology.orchestra.run.smithi070.stdout:2078 trunc from 0xe5c523f to 0x398f860 2024-09-16T11:43:48.383 INFO:teuthology.orchestra.run.smithi070.stdout:2079 write 0xc844afc thru 0xc851bda (0xd0df bytes) 2024-09-16T11:43:48.385 INFO:teuthology.orchestra.run.smithi070.stdout:2080 read 0xbed4a48 thru 0xbed7b9c (0x3155 bytes) 2024-09-16T11:43:48.392 INFO:teuthology.orchestra.run.smithi070.stdout:2081 write 0xd989716 thru 0xd99960a (0xfef5 bytes) 2024-09-16T11:43:48.395 INFO:teuthology.orchestra.run.smithi070.stdout:2082 punch from 0x2e2ae33 to 0x2e37c17, (0xcde4 bytes) 2024-09-16T11:43:48.395 INFO:teuthology.orchestra.run.smithi070.stdout:2083 write 0x945e617 thru 0x9467f16 (0x9900 bytes) 2024-09-16T11:43:48.400 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:48.400 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:48.407 INFO:teuthology.orchestra.run.smithi070.stdout:2087 trunc from 0xd99960b to 0x6e2233d 2024-09-16T11:43:48.424 INFO:teuthology.orchestra.run.smithi070.stdout:2088 write 0x9386abf thru 0x9392171 (0xb6b3 bytes) 2024-09-16T11:43:48.428 INFO:teuthology.orchestra.run.smithi070.stdout:2089 trunc from 0x9392172 to 0x8b0493c 2024-09-16T11:43:48.437 INFO:teuthology.orchestra.run.smithi070.stdout:2090 punch from 0x68420d3 to 0x6844c81, (0x2bae bytes) 2024-09-16T11:43:48.437 INFO:teuthology.orchestra.run.smithi070.stdout:2091 read 0x3d9d3fd thru 0x3da83dc (0xafe0 bytes) 2024-09-16T11:43:48.437 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:48.437 INFO:teuthology.orchestra.run.smithi070.stdout:2094 read 0x4a09461 thru 0x4a17380 (0xdf20 bytes) 2024-09-16T11:43:48.437 INFO:teuthology.orchestra.run.smithi070.stdout:2095 write 0xd5e267 thru 0xd66e04 (0x8b9e bytes) 2024-09-16T11:43:48.438 INFO:teuthology.orchestra.run.smithi070.stdout:2096 read 0x293fcce thru 0x2940e1a (0x114d bytes) 2024-09-16T11:43:48.438 INFO:teuthology.orchestra.run.smithi070.stdout:2097 read 0xdee7e0 thru 0xdfa288 (0xbaa9 bytes) 2024-09-16T11:43:48.442 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:48.439+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:48.444 INFO:teuthology.orchestra.run.smithi070.stdout:2098 trunc from 0x8b0493c to 0xd3ae2aa 2024-09-16T11:43:48.446 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:48.446 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:48.449 INFO:teuthology.orchestra.run.smithi070.stdout:2099 read 0x25494ae thru 0x2551bce (0x8721 bytes) 2024-09-16T11:43:48.449 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:48.457 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:43:48.459 INFO:teuthology.orchestra.run.smithi070.stdout:2104 trunc from 0xd3ae2aa to 0x4ffd19c 2024-09-16T11:43:48.464 INFO:teuthology.orchestra.run.smithi070.stdout:2105 write 0x397010e thru 0x39727d0 (0x26c3 bytes) 2024-09-16T11:43:48.467 INFO:teuthology.orchestra.run.smithi070.stdout:2106 clone 9 order 21 su 524288 sc 4 2024-09-16T11:43:48.542 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:48.541+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:48.557 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:48.557 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:48.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:43:48.649 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:48.647+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:48.704 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:48.704 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:48.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:43:48.743 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:48.742+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T11:43:48.844 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:48.842+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:48.856 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:48.856 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:48.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:43:48.884 INFO:teuthology.orchestra.run.smithi070.stdout:leaving image image_client.0-clone8 intact 2024-09-16T11:43:48.945 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:48.944+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:48.958 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-16T11:43:48.990 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:48.990 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:49.002 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:43:49.046 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:49.044+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:49.106 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:49.107 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:49.115 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:43:49.147 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:49.145+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:49.183 INFO:teuthology.orchestra.run.smithi070.stdout:2107 writesame 0xc238baf thru 0xc23947d data_size 0xb(0x8cf bytes) 2024-09-16T11:43:49.183 INFO:teuthology.orchestra.run.smithi070.stdout:2110 read 0x19b501f thru 0x19b652c (0x150e bytes) 2024-09-16T11:43:49.184 INFO:teuthology.orchestra.run.smithi070.stdout:2111 read 0x5bf3048 thru 0x5bfc09e (0x9057 bytes) 2024-09-16T11:43:49.184 INFO:teuthology.orchestra.run.smithi070.stdout:2112 read 0x2b65fac thru 0x2b66e49 (0xe9e bytes) 2024-09-16T11:43:49.184 INFO:teuthology.orchestra.run.smithi070.stdout:2113 punch from 0x5a2dada to 0x5a2daf5, (0x1b bytes) 2024-09-16T11:43:49.184 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.184 INFO:teuthology.orchestra.run.smithi070.stdout:2118 read 0x75fb289 thru 0x7605f84 (0xacfc bytes) 2024-09-16T11:43:49.195 INFO:teuthology.orchestra.run.smithi070.stdout:2119 trunc from 0xc23947e to 0x21354e3 2024-09-16T11:43:49.205 INFO:teuthology.orchestra.run.smithi070.stdout:2120 punch from 0xee6555 to 0xeef42b, (0x8ed6 bytes) 2024-09-16T11:43:49.206 INFO:teuthology.orchestra.run.smithi070.stdout:2121 trunc from 0x21354e3 to 0x3286850 2024-09-16T11:43:49.211 INFO:teuthology.orchestra.run.smithi070.stdout:2122 read 0x5e7fcc thru 0x5f247e (0xa4b3 bytes) 2024-09-16T11:43:49.212 INFO:teuthology.orchestra.run.smithi070.stdout:2124 punch from 0x262ce32 to 0x263025e, (0x342c bytes) 2024-09-16T11:43:49.212 INFO:teuthology.orchestra.run.smithi070.stdout:2126 punch from 0x421efb to 0x421f26, (0x2b bytes) 2024-09-16T11:43:49.212 INFO:teuthology.orchestra.run.smithi070.stdout:2127 punch from 0x25dd81f to 0x25e5937, (0x8118 bytes) 2024-09-16T11:43:49.227 INFO:teuthology.orchestra.run.smithi070.stdout:2128 writesame 0xb28b6cc thru 0xb296c3a data_size 0x1(0xb56f bytes) 2024-09-16T11:43:49.227 INFO:teuthology.orchestra.run.smithi070.stdout:2129 punch from 0x9112aa3 to 0x9119f89, (0x74e6 bytes) 2024-09-16T11:43:49.227 INFO:teuthology.orchestra.run.smithi070.stdout:2130 read 0x8571ff0 thru 0x8574685 (0x2696 bytes) 2024-09-16T11:43:49.229 INFO:teuthology.orchestra.run.smithi070.stdout:2131 trunc from 0xb296c3b to 0xda616f2 2024-09-16T11:43:49.234 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.234 INFO:teuthology.orchestra.run.smithi070.stdout:2134 read 0x4f2b26 thru 0x501e0f (0xf2ea bytes) 2024-09-16T11:43:49.235 INFO:teuthology.orchestra.run.smithi070.stdout:2135 write 0x9b766cd thru 0x9b80c52 (0xa586 bytes) 2024-09-16T11:43:49.240 INFO:teuthology.orchestra.run.smithi070.stdout:2136 write 0x674c9a8 thru 0x675ab8a (0xe1e3 bytes) 2024-09-16T11:43:49.243 INFO:teuthology.orchestra.run.smithi070.stdout:2137 write 0x8152720 thru 0x8155143 (0x2a24 bytes) 2024-09-16T11:43:49.247 INFO:teuthology.orchestra.run.smithi070.stdout:2138 trunc from 0xda616f2 to 0xe3ab903 2024-09-16T11:43:49.247 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:49.246+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:49.252 INFO:teuthology.orchestra.run.smithi070.stdout:2139 punch from 0x394f066 to 0x3959511, (0xa4ab bytes) 2024-09-16T11:43:49.252 INFO:teuthology.orchestra.run.smithi070.stdout:2140 writesame 0x4eb0cae thru 0x4eb9076 data_size 0x1(0x83c9 bytes) 2024-09-16T11:43:49.264 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:49.264 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:49.269 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.269 INFO:teuthology.orchestra.run.smithi070.stdout:2144 read 0x2c8935a thru 0x2c8fef5 (0x6b9c bytes) 2024-09-16T11:43:49.270 INFO:teuthology.orchestra.run.smithi070.stdout:2145 read 0x9182998 thru 0x918f9a9 (0xd012 bytes) 2024-09-16T11:43:49.270 INFO:teuthology.orchestra.run.smithi070.stdout:2146 write 0x8b124ab thru 0x8b1506f (0x2bc5 bytes) 2024-09-16T11:43:49.273 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:43:49.273 INFO:teuthology.orchestra.run.smithi070.stdout:2147 write 0x1d071b7 thru 0x1d08b67 (0x19b1 bytes) 2024-09-16T11:43:49.276 INFO:teuthology.orchestra.run.smithi070.stdout:2148 writesame 0xc4c7647 thru 0xc4cc9a7 data_size 0x5(0x5361 bytes) 2024-09-16T11:43:49.279 INFO:teuthology.orchestra.run.smithi070.stdout:2150 read 0xa12df8a thru 0xa13cb0b (0xeb82 bytes) 2024-09-16T11:43:49.280 INFO:teuthology.orchestra.run.smithi070.stdout:2152 read 0x6e1a78 thru 0x6e3685 (0x1c0e bytes) 2024-09-16T11:43:49.280 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.280 INFO:teuthology.orchestra.run.smithi070.stdout:2154 writesame 0xbae02df thru 0xbaed948 data_size 0x37(0xd66a bytes) 2024-09-16T11:43:49.284 INFO:teuthology.orchestra.run.smithi070.stdout:2157 read 0x42850e0 thru 0x428d359 (0x827a bytes) 2024-09-16T11:43:49.284 INFO:teuthology.orchestra.run.smithi070.stdout:2159 write 0x9908754 thru 0x990ab41 (0x23ee bytes) 2024-09-16T11:43:49.285 INFO:teuthology.orchestra.run.smithi070.stdout:2161 punch from 0x68753fd to 0x6878888, (0x348b bytes) 2024-09-16T11:43:49.285 INFO:teuthology.orchestra.run.smithi070.stdout:2163 writesame 0x43202b thru 0x43435c data_size 0x11(0x2332 bytes) 2024-09-16T11:43:49.290 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.290 INFO:teuthology.orchestra.run.smithi070.stdout:2165 read 0xd90739a thru 0xd90fe9a (0x8b01 bytes) 2024-09-16T11:43:49.290 INFO:teuthology.orchestra.run.smithi070.stdout:2166 writesame 0xbedb3e7 thru 0xbee2bd2 data_size 0x32(0x77ec bytes) 2024-09-16T11:43:49.292 INFO:teuthology.orchestra.run.smithi070.stdout:2167 writesame 0x5626dec thru 0x5631f4b data_size 0x30(0xb160 bytes) 2024-09-16T11:43:49.295 INFO:teuthology.orchestra.run.smithi070.stdout:2168 read 0xcbab810 thru 0xcbabd3f (0x530 bytes) 2024-09-16T11:43:49.295 INFO:teuthology.orchestra.run.smithi070.stdout:2169 read 0x9f9eaa2 thru 0x9fa1231 (0x2790 bytes) 2024-09-16T11:43:49.295 INFO:teuthology.orchestra.run.smithi070.stdout:2170 write 0x49ee494 thru 0x49f8dcb (0xa938 bytes) 2024-09-16T11:43:49.298 INFO:teuthology.orchestra.run.smithi070.stdout:2171 write 0xb983c9b thru 0xb988b7f (0x4ee5 bytes) 2024-09-16T11:43:49.315 INFO:teuthology.orchestra.run.smithi070.stdout:2173 trunc from 0xe3ab903 to 0x29ebe8d 2024-09-16T11:43:49.326 INFO:teuthology.orchestra.run.smithi070.stdout:2174 read 0x170baa6 thru 0x17116da (0x5c35 bytes) 2024-09-16T11:43:49.339 INFO:teuthology.orchestra.run.smithi070.stdout:2175 write 0x8e1e819 thru 0x8e2a545 (0xbd2d bytes) 2024-09-16T11:43:49.342 INFO:teuthology.orchestra.run.smithi070.stdout:2176 writesame 0x1d80697 thru 0x1d8d13e data_size 0x14(0xcaa8 bytes) 2024-09-16T11:43:49.346 INFO:teuthology.orchestra.run.smithi070.stdout:2177 write 0x171d050 thru 0x1721381 (0x4332 bytes) 2024-09-16T11:43:49.348 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:49.347+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:49.358 INFO:teuthology.orchestra.run.smithi070.stdout:2179 write 0xc20e617 thru 0xc210056 (0x1a40 bytes) 2024-09-16T11:43:49.371 INFO:teuthology.orchestra.run.smithi070.stdout:2180 trunc from 0xc210057 to 0x4676a62 2024-09-16T11:43:49.381 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:49.381 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:49.382 INFO:teuthology.orchestra.run.smithi070.stdout:2181 punch from 0x827eb to 0x8fcc1, (0xd4d6 bytes) 2024-09-16T11:43:49.382 INFO:teuthology.orchestra.run.smithi070.stdout:2183 read 0x33c759e thru 0x33cfb02 (0x8565 bytes) 2024-09-16T11:43:49.382 INFO:teuthology.orchestra.run.smithi070.stdout:2184 read 0x403a334 thru 0x403e728 (0x43f5 bytes) 2024-09-16T11:43:49.383 INFO:teuthology.orchestra.run.smithi070.stdout:2185 trunc from 0x4676a62 to 0x5487d2d 2024-09-16T11:43:49.388 INFO:teuthology.orchestra.run.smithi070.stdout:2187 read 0x32f6d4e thru 0x32f987c (0x2b2f bytes) 2024-09-16T11:43:49.388 INFO:teuthology.orchestra.run.smithi070.stdout:2189 write 0x40f1549 thru 0x40f970b (0x81c3 bytes) 2024-09-16T11:43:49.391 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:43:49.391 INFO:teuthology.orchestra.run.smithi070.stdout:2192 read 0x199c794 thru 0x199f773 (0x2fe0 bytes) 2024-09-16T11:43:49.403 INFO:teuthology.orchestra.run.smithi070.stdout:2193 writesame 0x9cccb78 thru 0x9ccdb4b data_size 0x2(0xfd4 bytes) 2024-09-16T11:43:49.403 INFO:teuthology.orchestra.run.smithi070.stdout:2194 read 0x467d09a thru 0x4688a86 (0xb9ed bytes) 2024-09-16T11:43:49.403 INFO:teuthology.orchestra.run.smithi070.stdout:2195 read 0x39ffe4d thru 0x3a0b288 (0xb43c bytes) 2024-09-16T11:43:49.403 INFO:teuthology.orchestra.run.smithi070.stdout:2199 read 0x411c33c thru 0x4125813 (0x94d8 bytes) 2024-09-16T11:43:49.403 INFO:teuthology.orchestra.run.smithi070.stdout:2202 punch from 0x7e35fed to 0x7e41aa9, (0xbabc bytes) 2024-09-16T11:43:49.414 INFO:teuthology.orchestra.run.smithi070.stdout:2203 writesame 0xebe896c thru 0xebe8f4d data_size 0x3(0x5e2 bytes) 2024-09-16T11:43:49.415 INFO:teuthology.orchestra.run.smithi070.stdout:2205 writesame 0x9e27739 thru 0x9e32735 data_size 0x1(0xaffd bytes) 2024-09-16T11:43:49.442 INFO:teuthology.orchestra.run.smithi070.stdout:2206 write 0xee407d4 thru 0xee50338 (0xfb65 bytes) 2024-09-16T11:43:49.449 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:49.447+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:49.452 INFO:teuthology.orchestra.run.smithi070.stdout:2209 trunc from 0xee50339 to 0x871c243 2024-09-16T11:43:49.461 INFO:teuthology.orchestra.run.smithi070.stdout:2210 writesame 0x40f9c29 thru 0x4104079 data_size 0x2f(0xa451 bytes) 2024-09-16T11:43:49.464 INFO:teuthology.orchestra.run.smithi070.stdout:2212 punch from 0x38e75d5 to 0x38effaf, (0x89da bytes) 2024-09-16T11:43:49.470 INFO:teuthology.orchestra.run.smithi070.stdout:2213 write 0x93aeda7 thru 0x93bc0bc (0xd316 bytes) 2024-09-16T11:43:49.474 INFO:teuthology.orchestra.run.smithi070.stdout:2214 punch from 0x5e0dd48 to 0x5e11af3, (0x3dab bytes) 2024-09-16T11:43:49.474 INFO:teuthology.orchestra.run.smithi070.stdout:2216 read 0x1ddd724 thru 0x1de6eb0 (0x978d bytes) 2024-09-16T11:43:49.474 INFO:teuthology.orchestra.run.smithi070.stdout:2220 read 0x81f6cb8 thru 0x820541e (0xe767 bytes) 2024-09-16T11:43:49.477 INFO:teuthology.orchestra.run.smithi070.stdout:2221 trunc from 0x93bc0bd to 0x6eab845 2024-09-16T11:43:49.487 INFO:teuthology.orchestra.run.smithi070.stdout:2222 read 0x1803d04 thru 0x180c7e5 (0x8ae2 bytes) 2024-09-16T11:43:49.488 INFO:teuthology.orchestra.run.smithi070.stdout:2223 trunc from 0x6eab845 to 0x625a2f4 2024-09-16T11:43:49.495 INFO:teuthology.orchestra.run.smithi070.stdout:2224 punch from 0x801bac to 0x805023, (0x3477 bytes) 2024-09-16T11:43:49.495 INFO:teuthology.orchestra.run.smithi070.stdout:2225 write 0x4333544 thru 0x4338b53 (0x5610 bytes) 2024-09-16T11:43:49.500 INFO:teuthology.orchestra.run.smithi070.stdout:2226 trunc from 0x625a2f4 to 0x94e2664 2024-09-16T11:43:49.507 INFO:teuthology.orchestra.run.smithi070.stdout:2227 trunc from 0x94e2664 to 0x8889b64 2024-09-16T11:43:49.513 INFO:teuthology.orchestra.run.smithi070.stdout:2228 write 0x7973cbd thru 0x7980595 (0xc8d9 bytes) 2024-09-16T11:43:49.517 INFO:teuthology.orchestra.run.smithi070.stdout:2229 read 0x8877b66 thru 0x887d879 (0x5d14 bytes) 2024-09-16T11:43:49.518 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.518 INFO:teuthology.orchestra.run.smithi070.stdout:2232 writesame 0x421a610 thru 0x421f15f data_size 0x28(0x4b50 bytes) 2024-09-16T11:43:49.521 INFO:teuthology.orchestra.run.smithi070.stdout:2233 trunc from 0x8889b64 to 0x9357e40 2024-09-16T11:43:49.532 INFO:teuthology.orchestra.run.smithi070.stdout:2234 write 0xa01390e thru 0xa0170e4 (0x37d7 bytes) 2024-09-16T11:43:49.535 INFO:teuthology.orchestra.run.smithi070.stdout:2235 write 0x8d11506 thru 0x8d1193e (0x439 bytes) 2024-09-16T11:43:49.538 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:49.538 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:49.543 INFO:teuthology.orchestra.run.smithi070.stdout:2236 trunc from 0xa0170e5 to 0xe791eed 2024-09-16T11:43:49.547 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:43:49.550 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:49.547+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:49.553 INFO:teuthology.orchestra.run.smithi070.stdout:2237 trunc from 0xe791eed to 0xbd5d830 2024-09-16T11:43:49.558 INFO:teuthology.orchestra.run.smithi070.stdout:2240 read 0x7ef37e0 thru 0x7efe72b (0xaf4c bytes) 2024-09-16T11:43:49.558 INFO:teuthology.orchestra.run.smithi070.stdout:2241 read 0x35f371e thru 0x3600369 (0xcc4c bytes) 2024-09-16T11:43:49.558 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.558 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.562 INFO:teuthology.orchestra.run.smithi070.stdout:2246 trunc from 0xbd5d830 to 0x8110d54 2024-09-16T11:43:49.582 INFO:teuthology.orchestra.run.smithi070.stdout:2247 writesame 0xc3e5fd1 thru 0xc3e8c74 data_size 0x2(0x2ca4 bytes) 2024-09-16T11:43:49.582 INFO:teuthology.orchestra.run.smithi070.stdout:2248 punch from 0x8465c00 to 0x846b647, (0x5a47 bytes) 2024-09-16T11:43:49.583 INFO:teuthology.orchestra.run.smithi070.stdout:2249 write 0x77d6db9 thru 0x77e6778 (0xf9c0 bytes) 2024-09-16T11:43:49.585 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.586 INFO:teuthology.orchestra.run.smithi070.stdout:2251 read 0x1eb109e thru 0x1ebbe18 (0xad7b bytes) 2024-09-16T11:43:49.586 INFO:teuthology.orchestra.run.smithi070.stdout:2252 read 0x76af67e thru 0x76b4c69 (0x55ec bytes) 2024-09-16T11:43:49.586 INFO:teuthology.orchestra.run.smithi070.stdout:2253 writesame 0x8a2b9ad thru 0x8a2f258 data_size 0x34(0x38ac bytes) 2024-09-16T11:43:49.589 INFO:teuthology.orchestra.run.smithi070.stdout:2254 read 0xa9e0604 thru 0xa9ede22 (0xd81f bytes) 2024-09-16T11:43:49.599 INFO:teuthology.orchestra.run.smithi070.stdout:2256 trunc from 0xc3e8c75 to 0x3491539 2024-09-16T11:43:49.610 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.625 INFO:teuthology.orchestra.run.smithi070.stdout:2258 writesame 0xb7a0cab thru 0xb7a6462 data_size 0xe(0x57b8 bytes) 2024-09-16T11:43:49.631 INFO:teuthology.orchestra.run.smithi070.stdout:2259 write 0xc5f41b2 thru 0xc5fbbee (0x7a3d bytes) 2024-09-16T11:43:49.635 INFO:teuthology.orchestra.run.smithi070.stdout:2260 read 0x5015357 thru 0x501adf5 (0x5a9f bytes) 2024-09-16T11:43:49.635 INFO:teuthology.orchestra.run.smithi070.stdout:2261 write 0x4ec7a52 thru 0x4ecf21a (0x77c9 bytes) 2024-09-16T11:43:49.639 INFO:teuthology.orchestra.run.smithi070.stdout:2262 read 0x4e76349 thru 0x4e776d6 (0x138e bytes) 2024-09-16T11:43:49.639 INFO:teuthology.orchestra.run.smithi070.stdout:2263 write 0xa784774 thru 0xa79048d (0xbd1a bytes) 2024-09-16T11:43:49.643 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.644 INFO:teuthology.orchestra.run.smithi070.stdout:2266 read 0xdc1d24 thru 0xdc9b6b (0x7e48 bytes) 2024-09-16T11:43:49.644 INFO:teuthology.orchestra.run.smithi070.stdout:2268 writesame 0x6adbd80 thru 0x6ae05d2 data_size 0x23(0x4853 bytes) 2024-09-16T11:43:49.647 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.647 INFO:teuthology.orchestra.run.smithi070.stdout:2271 writesame 0xbd659cc thru 0xbd73b29 data_size 0x5b(0xe15e bytes) 2024-09-16T11:43:49.650 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:49.648+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:49.650 INFO:teuthology.orchestra.run.smithi070.stdout:2272 read 0x258f64a thru 0x2593734 (0x40eb bytes) 2024-09-16T11:43:49.654 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:49.654 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:49.658 INFO:teuthology.orchestra.run.smithi070.stdout:2274 write 0xe5c0fac thru 0xe5c71ba (0x620f bytes) 2024-09-16T11:43:49.665 INFO:teuthology.orchestra.run.smithi070.stdout:2275 write 0x8f72daa thru 0x8f81fba (0xf211 bytes) 2024-09-16T11:43:49.665 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:43:49.667 INFO:teuthology.orchestra.run.smithi070.stdout:2277 write 0xaf6737f thru 0xaf699bc (0x263e bytes) 2024-09-16T11:43:49.675 INFO:teuthology.orchestra.run.smithi070.stdout:2278 write 0xecb59d9 thru 0xecb59f1 (0x19 bytes) 2024-09-16T11:43:49.678 INFO:teuthology.orchestra.run.smithi070.stdout:2281 read 0xd01723 thru 0xd10e4f (0xf72d bytes) 2024-09-16T11:43:49.679 INFO:teuthology.orchestra.run.smithi070.stdout:2283 read 0xbe2261a thru 0xbe242c2 (0x1ca9 bytes) 2024-09-16T11:43:49.694 INFO:teuthology.orchestra.run.smithi070.stdout:2285 trunc from 0xecb59f2 to 0x1f9db87 2024-09-16T11:43:49.706 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.706 INFO:teuthology.orchestra.run.smithi070.stdout:2287 read 0xca66ac thru 0xcb5179 (0xeace bytes) 2024-09-16T11:43:49.714 INFO:teuthology.orchestra.run.smithi070.stdout:2288 writesame 0x3c162aa thru 0x3c201fb data_size 0x1(0x9f52 bytes) 2024-09-16T11:43:49.715 INFO:teuthology.orchestra.run.smithi070.stdout:2289 read 0x1d048ea thru 0x1d07ce6 (0x33fd bytes) 2024-09-16T11:43:49.718 INFO:teuthology.orchestra.run.smithi070.stdout:2290 trunc from 0x3c201fc to 0x106d3e9 2024-09-16T11:43:49.748 INFO:teuthology.orchestra.run.smithi070.stdout:2291 write 0x5b64fce thru 0x5b73b8e (0xebc1 bytes) 2024-09-16T11:43:49.751 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:49.750+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:49.751 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.760 INFO:teuthology.orchestra.run.smithi070.stdout:2293 writesame 0x8a15882 thru 0x8a20c94 data_size 0x1(0xb413 bytes) 2024-09-16T11:43:49.766 INFO:teuthology.orchestra.run.smithi070.stdout:2294 write 0x9582c6e thru 0x958ca8b (0x9e1e bytes) 2024-09-16T11:43:49.770 INFO:teuthology.orchestra.run.smithi070.stdout:2295 write 0x5288e9b thru 0x529695a (0xdac0 bytes) 2024-09-16T11:43:49.771 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:49.771 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:49.773 INFO:teuthology.orchestra.run.smithi070.stdout:2296 write 0x78d96a6 thru 0x78d9b88 (0x4e3 bytes) 2024-09-16T11:43:49.781 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:43:49.782 INFO:teuthology.orchestra.run.smithi070.stdout:2297 trunc from 0x958ca8c to 0x4e42122 2024-09-16T11:43:49.804 INFO:teuthology.orchestra.run.smithi070.stdout:2298 write 0x91d604b thru 0x91e4f32 (0xeee8 bytes) 2024-09-16T11:43:49.815 INFO:teuthology.orchestra.run.smithi070.stdout:2299 writesame 0xaf35465 thru 0xaf3c2ac data_size 0x4(0x6e48 bytes) 2024-09-16T11:43:49.825 INFO:teuthology.orchestra.run.smithi070.stdout:2300 writesame 0xe16c24b thru 0xe179411 data_size 0x11(0xd1c7 bytes) 2024-09-16T11:43:49.825 INFO:teuthology.orchestra.run.smithi070.stdout:2301 read 0x96244a8 thru 0x962905e (0x4bb7 bytes) 2024-09-16T11:43:49.825 INFO:teuthology.orchestra.run.smithi070.stdout:2302 read 0x196427e thru 0x196b309 (0x708c bytes) 2024-09-16T11:43:49.825 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:49.825 INFO:teuthology.orchestra.run.smithi070.stdout:2304 write 0x4f287a6 thru 0x4f31efc (0x9757 bytes) 2024-09-16T11:43:49.828 INFO:teuthology.orchestra.run.smithi070.stdout:2305 write 0xa00f3e4 thru 0xa01951f (0xa13c bytes) 2024-09-16T11:43:49.831 INFO:teuthology.orchestra.run.smithi070.stdout:2307 clone 10 order 24 su 4194304 sc 10 2024-09-16T11:43:49.865 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:49.864+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:49.921 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:49.921 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:49.929 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:43:49.953 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:49.952+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:50.054 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:50.053+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:50.067 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:50.067 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:50.076 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:43:50.155 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:50.153+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:50.215 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:50.215 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:50.223 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:43:50.268 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:50.267+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:50.321 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:50.321 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:50.329 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:43:50.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:43:50.370 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:50.369+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:50.458 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:50.456+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:50.467 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:50.467 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:50.475 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:43:50.478 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:43:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:43:50.493 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:43:50.493 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:43:50.508 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:43:50.508 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:43:50.522 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:43:50.522 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:43:50.537 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:43:50.538 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:43:50.553 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:43:50.553 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:43:50.560 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:50.559+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:50.568 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:43:50.568 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:43:50.583 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:43:50.583 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:43:50.615 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:50.615 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:50.616 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:43:50.616 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:50.624 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:43:50.665 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:50.664+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:50.760 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:50.759+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:50.762 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:50.762 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:50.762 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:50.763 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:50.763 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:50.771 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:43:50.846 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:43:50.861 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:50.860+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:50.900 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone9 from 0xe179412 (overlap 0x106d3e9) to 0x1a4864 2024-09-16T11:43:50.924 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:50.924 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:43:50.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:43:50.924 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:43:50.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.924 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:50.924 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.718083+0000", 2024-09-16T11:43:50.924 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.202714053000001, 2024-09-16T11:43:50.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061853209999999997, 2024-09-16T11:43:50.925 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:50.925 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:50.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:50.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:50.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:50.925 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 22 2024-09-16T11:43:50.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.925 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:50.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:50.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718083+0000", 2024-09-16T11:43:50.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718083+0000", 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718084+0000", 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:50.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718153+0000", 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9362000000000004e-05 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718159+0000", 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3060000000000003e-06 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718164+0000", 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3900000000000003e-06 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:50.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719004+0000", 2024-09-16T11:43:50.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00084073899999999996 2024-09-16T11:43:50.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:50.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719050+0000", 2024-09-16T11:43:50.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5124999999999999e-05 2024-09-16T11:43:50.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:50.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719125+0000", 2024-09-16T11:43:50.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5914999999999994e-05 2024-09-16T11:43:50.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:50.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720731+0000", 2024-09-16T11:43:50.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016052589999999999 2024-09-16T11:43:50.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:50.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724213+0000", 2024-09-16T11:43:50.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034819180000000001 2024-09-16T11:43:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724245+0000", 2024-09-16T11:43:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2725999999999998e-05 2024-09-16T11:43:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724268+0000", 2024-09-16T11:43:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2954000000000001e-05 2024-09-16T11:43:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.931 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:50.931 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.718169+0000", 2024-09-16T11:43:50.931 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.202628509, 2024-09-16T11:43:50.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061430139999999996, 2024-09-16T11:43:50.931 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:50.931 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:50.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:50.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:50.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:50.931 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 23 2024-09-16T11:43:50.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.932 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:50.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:50.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718169+0000", 2024-09-16T11:43:50.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:50.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718169+0000", 2024-09-16T11:43:50.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718169+0000", 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718227+0000", 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7714999999999999e-05 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718233+0000", 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9320000000000001e-06 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:50.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718236+0000", 2024-09-16T11:43:50.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:43:50.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:50.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719452+0000", 2024-09-16T11:43:50.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001215673 2024-09-16T11:43:50.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:50.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719501+0000", 2024-09-16T11:43:50.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9283000000000002e-05 2024-09-16T11:43:50.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:50.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719584+0000", 2024-09-16T11:43:50.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3182000000000004e-05 2024-09-16T11:43:50.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:50.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720759+0000", 2024-09-16T11:43:50.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011749810000000001 2024-09-16T11:43:50.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:50.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724273+0000", 2024-09-16T11:43:50.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035134649999999999 2024-09-16T11:43:50.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:50.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724294+0000", 2024-09-16T11:43:50.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1158999999999999e-05 2024-09-16T11:43:50.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724312+0000", 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740999999999998e-05 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.718240+0000", 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.202557214000002, 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061176770000000002, 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:50.937 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:50.938 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:50.938 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:50.938 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:50.938 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 24 2024-09-16T11:43:50.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.938 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:50.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:50.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718240+0000", 2024-09-16T11:43:50.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:50.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718240+0000", 2024-09-16T11:43:50.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:50.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718240+0000", 2024-09-16T11:43:50.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:43:50.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:50.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718296+0000", 2024-09-16T11:43:50.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5365999999999998e-05 2024-09-16T11:43:50.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:50.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718301+0000", 2024-09-16T11:43:50.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5219999999999998e-06 2024-09-16T11:43:50.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:50.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718308+0000", 2024-09-16T11:43:50.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5440000000000004e-06 2024-09-16T11:43:50.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:50.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.719993+0000", 2024-09-16T11:43:50.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001685717 2024-09-16T11:43:50.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:50.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720044+0000", 2024-09-16T11:43:50.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1007000000000001e-05 2024-09-16T11:43:50.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:50.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720123+0000", 2024-09-16T11:43:50.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8714999999999994e-05 2024-09-16T11:43:50.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:50.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720918+0000", 2024-09-16T11:43:50.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00079541099999999999 2024-09-16T11:43:50.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:50.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724319+0000", 2024-09-16T11:43:50.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034004740000000001 2024-09-16T11:43:50.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:50.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724340+0000", 2024-09-16T11:43:50.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1131000000000001e-05 2024-09-16T11:43:50.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:50.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724358+0000", 2024-09-16T11:43:50.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7458000000000001e-05 2024-09-16T11:43:50.944 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.944 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:50.944 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.944 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:50.944 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.718312+0000", 2024-09-16T11:43:50.944 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.202485500000002, 2024-09-16T11:43:50.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060983740000000002, 2024-09-16T11:43:50.944 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:50.944 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:50.945 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:50.945 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:50.945 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:50.945 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-16T11:43:50.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.945 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:50.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:50.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718312+0000", 2024-09-16T11:43:50.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:50.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718312+0000", 2024-09-16T11:43:50.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:50.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718312+0000", 2024-09-16T11:43:50.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:43:50.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:50.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718314+0000", 2024-09-16T11:43:50.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:43:50.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:50.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718318+0000", 2024-09-16T11:43:50.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3970000000000004e-06 2024-09-16T11:43:50.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:50.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.718321+0000", 2024-09-16T11:43:50.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4219999999999999e-06 2024-09-16T11:43:50.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:50.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720425+0000", 2024-09-16T11:43:50.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021042690000000002 2024-09-16T11:43:50.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:50.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720474+0000", 2024-09-16T11:43:50.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8449000000000003e-05 2024-09-16T11:43:50.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:50.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.720548+0000", 2024-09-16T11:43:50.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4276999999999994e-05 2024-09-16T11:43:50.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:50.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.721085+0000", 2024-09-16T11:43:50.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000537252 2024-09-16T11:43:50.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:50.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724361+0000", 2024-09-16T11:43:50.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0032759289999999999 2024-09-16T11:43:50.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:50.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724393+0000", 2024-09-16T11:43:50.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1930000000000001e-05 2024-09-16T11:43:50.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:50.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.724410+0000", 2024-09-16T11:43:50.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7002000000000001e-05 2024-09-16T11:43:50.951 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.951 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:50.951 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.951 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:50.952 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:43:50.952 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.000773348999999, 2024-09-16T11:43:50.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:43:50.952 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:50.952 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:50.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:50.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:50.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:50.952 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:43:50.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.953 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:50.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:50.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:43:50.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:50.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:43:50.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:50.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:43:50.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:43:50.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:50.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:43:50.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:43:50.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:50.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:43:50.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:43:50.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:50.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:43:50.958 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.000727656999999, 2024-09-16T11:43:50.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:43:50.959 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:50.959 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:50.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:50.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:50.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:50.959 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:43:50.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.959 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:50.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:50.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:43:50.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:50.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:43:50.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:50.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:43:50.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:43:50.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:50.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:43:50.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:43:50.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:50.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:43:50.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:43:50.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:50.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:43:50.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:43:50.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:50.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:43:50.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:43:50.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:50.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:43:50.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:43:50.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:50.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:43:50.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:43:50.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:50.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:43:50.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:43:50.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:50.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:43:50.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:43:50.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:50.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:43:50.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:43:50.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:50.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:43:50.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:43:50.965 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.965 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:50.965 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.965 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:50.966 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:43:50.966 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.000670726999999, 2024-09-16T11:43:50.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:43:50.966 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:50.966 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:50.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:50.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:50.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:50.966 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:43:50.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.966 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:50.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:50.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:43:50.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:50.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:43:50.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:50.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:43:50.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:43:50.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:50.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:43:50.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:43:50.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:50.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:43:50.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:43:50.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:50.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:43:50.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:43:50.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:50.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:43:50.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:43:50.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:50.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:43:50.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:43:50.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:50.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:43:50.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:43:50.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:50.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:43:50.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:43:50.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:50.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:43:50.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:43:50.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:50.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:43:50.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:43:50.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:50.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:43:50.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:43:50.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:43:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.000645359000004, 2024-09-16T11:43:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:43:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:50.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:50.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:50.974 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:43:50.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.974 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:50.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:50.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:43:50.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:50.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:43:50.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:50.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:43:50.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:43:50.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:50.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:43:50.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:43:50.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:50.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:43:50.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:43:50.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:50.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:43:50.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:43:50.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:50.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:43:50.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:43:50.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:50.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:43:50.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:43:50.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:50.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:43:50.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:43:50.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:50.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:43:50.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:43:50.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:50.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:43:50.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:43:50.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:50.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:43:50.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:43:50.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:50.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:43:50.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:43:50.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.982 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:50.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.983 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:50.983 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:43:50.983 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.000593391000002, 2024-09-16T11:43:50.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:43:50.983 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:50.983 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:50.983 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:50.983 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:50.983 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:50.983 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:43:50.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.984 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:50.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:50.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:43:50.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:50.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:43:50.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:50.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:43:50.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:43:50.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:50.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:43:50.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:50.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:43:50.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:43:50.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:50.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:43:50.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:43:50.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:50.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:43:50.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:43:50.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:43:50.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:43:50.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:50.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:43:50.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:43:50.989 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.989 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:50.989 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.989 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:50.989 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:43:50.989 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.000512280999999, 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:50.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:43:50.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:50.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:43:50.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:43:50.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:50.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:43:50.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:43:50.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:50.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:43:50.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:50.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.995 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:50.996 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:50.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.996 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:50.996 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:43:50.996 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.000486303000002, 2024-09-16T11:43:50.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:43:50.996 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:50.996 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:50.996 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:50.996 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:50.996 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:50.996 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:43:50.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.997 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:50.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:50.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:43:50.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:50.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:43:50.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:50.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:43:50.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:43:50.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:50.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:43:50.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:43:50.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:50.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:43:50.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:43:50.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:50.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:50.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:51.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:43:51.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:43:51.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:51.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:43:51.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:43:51.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:51.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:43:51.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:43:51.002 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.002 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:51.002 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.002 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:51.002 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337703+0000", 2024-09-16T11:43:51.002 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.583093636000001, 2024-09-16T11:43:51.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053663890000000001, 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 70 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337703+0000", 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:51.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337703+0000", 2024-09-16T11:43:51.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:51.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337704+0000", 2024-09-16T11:43:51.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:43:51.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:51.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337706+0000", 2024-09-16T11:43:51.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:43:51.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:51.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337711+0000", 2024-09-16T11:43:51.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2900000000000002e-06 2024-09-16T11:43:51.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337714+0000", 2024-09-16T11:43:51.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3970000000000001e-06 2024-09-16T11:43:51.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:51.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341010+0000", 2024-09-16T11:43:51.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0032962220000000001 2024-09-16T11:43:51.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:51.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341092+0000", 2024-09-16T11:43:51.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1996999999999994e-05 2024-09-16T11:43:51.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:51.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341220+0000", 2024-09-16T11:43:51.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012852600000000001 2024-09-16T11:43:51.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.342994+0000", 2024-09-16T11:43:51.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001773279 2024-09-16T11:43:51.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:51.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343021+0000", 2024-09-16T11:43:51.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.758e-05 2024-09-16T11:43:51.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:51.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343049+0000", 2024-09-16T11:43:51.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8118000000000001e-05 2024-09-16T11:43:51.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:51.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343070+0000", 2024-09-16T11:43:51.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0330999999999999e-05 2024-09-16T11:43:51.008 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.008 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:51.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.009 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:51.009 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337716+0000", 2024-09-16T11:43:51.009 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.583080887000001, 2024-09-16T11:43:51.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054005590000000001, 2024-09-16T11:43:51.009 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:51.009 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.009 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:51.009 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:51.009 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:51.010 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 71 2024-09-16T11:43:51.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.010 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:51.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:51.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337716+0000", 2024-09-16T11:43:51.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:51.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337716+0000", 2024-09-16T11:43:51.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:51.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337716+0000", 2024-09-16T11:43:51.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-16T11:43:51.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:51.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337718+0000", 2024-09-16T11:43:51.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.657e-06 2024-09-16T11:43:51.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:51.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337723+0000", 2024-09-16T11:43:51.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3329999999999996e-06 2024-09-16T11:43:51.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337726+0000", 2024-09-16T11:43:51.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.039e-06 2024-09-16T11:43:51.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:51.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341844+0000", 2024-09-16T11:43:51.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041171699999999999 2024-09-16T11:43:51.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:51.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341901+0000", 2024-09-16T11:43:51.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7717999999999999e-05 2024-09-16T11:43:51.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:51.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.341974+0000", 2024-09-16T11:43:51.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2538000000000002e-05 2024-09-16T11:43:51.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343017+0000", 2024-09-16T11:43:51.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010429510000000001 2024-09-16T11:43:51.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:51.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343079+0000", 2024-09-16T11:43:51.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2045000000000007e-05 2024-09-16T11:43:51.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:51.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343100+0000", 2024-09-16T11:43:51.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1154000000000001e-05 2024-09-16T11:43:51.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:51.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343117+0000", 2024-09-16T11:43:51.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6741000000000001e-05 2024-09-16T11:43:51.015 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.015 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:51.015 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337730+0000", 2024-09-16T11:43:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.583066944000002, 2024-09-16T11:43:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063357409999999998, 2024-09-16T11:43:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-16T11:43:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.017 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:51.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:51.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337730+0000", 2024-09-16T11:43:51.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:51.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337730+0000", 2024-09-16T11:43:51.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:51.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337730+0000", 2024-09-16T11:43:51.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-16T11:43:51.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:51.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337785+0000", 2024-09-16T11:43:51.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4673000000000003e-05 2024-09-16T11:43:51.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:51.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337794+0000", 2024-09-16T11:43:51.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8829999999999995e-06 2024-09-16T11:43:51.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337798+0000", 2024-09-16T11:43:51.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1069999999999998e-06 2024-09-16T11:43:51.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:51.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.342223+0000", 2024-09-16T11:43:51.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.004425216 2024-09-16T11:43:51.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:51.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.342309+0000", 2024-09-16T11:43:51.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5424000000000001e-05 2024-09-16T11:43:51.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:51.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.342451+0000", 2024-09-16T11:43:51.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014200899999999999 2024-09-16T11:43:51.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343980+0000", 2024-09-16T11:43:51.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001529819 2024-09-16T11:43:51.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:51.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344021+0000", 2024-09-16T11:43:51.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0175000000000001e-05 2024-09-16T11:43:51.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:51.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344046+0000", 2024-09-16T11:43:51.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5677000000000002e-05 2024-09-16T11:43:51.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:51.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344066+0000", 2024-09-16T11:43:51.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9551999999999998e-05 2024-09-16T11:43:51.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.022 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:51.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.022 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:51.023 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337806+0000", 2024-09-16T11:43:51.023 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.582991069000002, 2024-09-16T11:43:51.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.006307857, 2024-09-16T11:43:51.023 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:51.023 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:51.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:51.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:51.023 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-16T11:43:51.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.024 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:51.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:51.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337806+0000", 2024-09-16T11:43:51.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:51.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337806+0000", 2024-09-16T11:43:51.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:51.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337807+0000", 2024-09-16T11:43:51.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7199999999999999e-07 2024-09-16T11:43:51.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:51.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337812+0000", 2024-09-16T11:43:51.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7259999999999999e-06 2024-09-16T11:43:51.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:51.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337820+0000", 2024-09-16T11:43:51.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9079999999999995e-06 2024-09-16T11:43:51.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337824+0000", 2024-09-16T11:43:51.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0740000000000003e-06 2024-09-16T11:43:51.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:51.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343154+0000", 2024-09-16T11:43:51.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053298640000000001 2024-09-16T11:43:51.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:51.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343203+0000", 2024-09-16T11:43:51.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8664e-05 2024-09-16T11:43:51.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:51.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343284+0000", 2024-09-16T11:43:51.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1211000000000006e-05 2024-09-16T11:43:51.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344015+0000", 2024-09-16T11:43:51.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00073082900000000001 2024-09-16T11:43:51.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:51.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344074+0000", 2024-09-16T11:43:51.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9416000000000002e-05 2024-09-16T11:43:51.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:51.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344095+0000", 2024-09-16T11:43:51.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0837999999999999e-05 2024-09-16T11:43:51.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:51.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344114+0000", 2024-09-16T11:43:51.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8754999999999999e-05 2024-09-16T11:43:51.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.030 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:51.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.030 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:51.030 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:43:51.030 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.582967678000003, 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:43:51.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:43:51.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:51.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:43:51.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:43:51.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:43:51.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:43:51.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:51.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:43:51.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:43:51.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:51.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:43:51.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:43:51.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:51.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:43:51.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:43:51.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:43:51.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:43:51.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:51.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:43:51.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:43:51.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:51.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:43:51.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:43:51.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:51.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:43:51.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:43:51.037 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.037 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:51.037 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.037 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:51.037 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:43:51.037 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.582945746999997, 2024-09-16T11:43:51.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:43:51.037 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:51.038 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:51.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:51.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:51.038 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:43:51.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.038 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:51.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:51.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:43:51.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:51.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:43:51.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:51.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:43:51.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:43:51.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:51.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:43:51.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:43:51.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:51.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:43:51.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:43:51.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:43:51.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:43:51.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:51.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:43:51.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:43:51.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:51.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:43:51.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:43:51.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:51.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:43:51.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:43:51.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:43:51.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:43:51.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:51.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:43:51.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:43:51.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:51.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:43:51.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:43:51.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:51.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:43:51.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:43:51.047 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.047 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:51.047 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.047 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:43:51.048 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:43:51.048 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.582868859000001, 2024-09-16T11:43:51.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:43:51.048 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:51.048 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:51.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:43:51.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:43:51.049 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:43:51.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.049 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:51.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:51.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:43:51.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:51.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:43:51.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:51.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:43:51.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:43:51.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:51.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:43:51.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:43:51.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:51.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:43:51.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:43:51.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:43:51.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:43:51.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:51.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:43:51.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:43:51.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:51.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:43:51.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:43:51.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:43:51.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:43:51.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:43:51.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:43:51.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:43:51.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:43:51.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:43:51.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:43:51.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:51.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:43:51.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:43:51.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:51.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:43:51.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:43:51.059 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.060 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:51.060 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.060 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:1165 4.17 e32/28 4:ebaef169:::rbd_data.11c17b8dc4e5.0000000000000006:head v 32'6, mlcod=32'4)", 2024-09-16T11:43:51.060 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.382439+0000", 2024-09-16T11:43:51.060 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.538357945, 2024-09-16T11:43:51.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059190919999999999, 2024-09-16T11:43:51.061 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:51.061 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.061 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:51.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:51.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382439+0000", 2024-09-16T11:43:51.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:51.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382439+0000", 2024-09-16T11:43:51.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:51.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382440+0000", 2024-09-16T11:43:51.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:43:51.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:51.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382445+0000", 2024-09-16T11:43:51.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9280000000000001e-06 2024-09-16T11:43:51.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:51.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382454+0000", 2024-09-16T11:43:51.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2040000000000003e-06 2024-09-16T11:43:51.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382459+0000", 2024-09-16T11:43:51.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2989999999999996e-06 2024-09-16T11:43:51.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:51.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382510+0000", 2024-09-16T11:43:51.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1066000000000002e-05 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382530+0000", 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9673999999999998e-05 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.388328+0000", 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057977920000000004 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:51.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.388358+0000", 2024-09-16T11:43:51.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0586999999999997e-05 2024-09-16T11:43:51.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.067 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:51.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.067 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:1167 4.11 e32/28 4:8a86a6ca:::rbd_data.11c17b8dc4e5.000000000000000a:head v 32'21, mlcod=32'19)", 2024-09-16T11:43:51.067 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:37.382770+0000", 2024-09-16T11:43:51.067 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.538027277999999, 2024-09-16T11:43:51.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0056292640000000001, 2024-09-16T11:43:51.067 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:51.067 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.067 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382770+0000", 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382770+0000", 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382770+0000", 2024-09-16T11:43:51.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9800000000000003e-07 2024-09-16T11:43:51.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:51.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382776+0000", 2024-09-16T11:43:51.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3009999999999997e-06 2024-09-16T11:43:51.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:51.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382788+0000", 2024-09-16T11:43:51.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2520999999999999e-05 2024-09-16T11:43:51.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382795+0000", 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9489999999999999e-06 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382819+0000", 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3649999999999999e-05 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.382832+0000", 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3519999999999999e-05 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.388364+0000", 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0055314359999999998 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:37.388399+0000", 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5188999999999998e-05 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:51.071 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:51.073 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-16T11:43:51.073 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.073 INFO:teuthology.orchestra.run.smithi070.stdout:2309 write 0x753c2dd thru 0x7545844 (0x9568 bytes) 2024-09-16T11:43:51.074 INFO:teuthology.orchestra.run.smithi070.stdout:2312 read 0x7872ff8 thru 0x7878f59 (0x5f62 bytes) 2024-09-16T11:43:51.074 INFO:teuthology.orchestra.run.smithi070.stdout:2313 read 0xd91e287 thru 0xd92ab41 (0xc8bb bytes) 2024-09-16T11:43:51.074 INFO:teuthology.orchestra.run.smithi070.stdout:2314 write 0xf6f555 thru 0xf799f0 (0xa49c bytes) 2024-09-16T11:43:51.074 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:51.071+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:51.074 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:43:51.081 INFO:teuthology.orchestra.run.smithi070.stdout:2315 writesame 0x247ea59 thru 0x2487323 data_size 0x9(0x88cb bytes) 2024-09-16T11:43:51.084 INFO:teuthology.orchestra.run.smithi070.stdout:2316 flatten 2024-09-16T11:43:51.097 INFO:teuthology.orchestra.run.smithi070.stdout:2317 read 0xc62f046 thru 0xc63c5ed (0xd5a8 bytes) 2024-09-16T11:43:51.097 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.097 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.097 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.097 INFO:teuthology.orchestra.run.smithi070.stdout:2328 write 0x14b2771 thru 0x14be161 (0xb9f1 bytes) 2024-09-16T11:43:51.101 INFO:teuthology.orchestra.run.smithi070.stdout:2329 punch from 0xa5ba968 to 0xa5ca435, (0xfacd bytes) 2024-09-16T11:43:51.101 INFO:teuthology.orchestra.run.smithi070.stdout:2330 write 0xd1ec388 thru 0xd1f6599 (0xa212 bytes) 2024-09-16T11:43:51.103 INFO:teuthology.orchestra.run.smithi070.stdout:2333 writesame 0x263b791 thru 0x264b0ce data_size 0x3d(0xf93e bytes) 2024-09-16T11:43:51.105 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.109 INFO:teuthology.orchestra.run.smithi070.stdout:2335 write 0xe2046f5 thru 0xe213e7c (0xf788 bytes) 2024-09-16T11:43:51.112 INFO:teuthology.orchestra.run.smithi070.stdout:2336 read 0x91fe01d thru 0x92057d1 (0x77b5 bytes) 2024-09-16T11:43:51.112 INFO:teuthology.orchestra.run.smithi070.stdout:2339 read 0xb7e3349 thru 0xb7f32fb (0xffb3 bytes) 2024-09-16T11:43:51.113 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.113 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.113 INFO:teuthology.orchestra.run.smithi070.stdout:2343 punch from 0xb36684a to 0xb3755b4, (0xed6a bytes) 2024-09-16T11:43:51.113 INFO:teuthology.orchestra.run.smithi070.stdout:2344 writesame 0x59b8820 thru 0x59bcde8 data_size 0xf(0x45c9 bytes) 2024-09-16T11:43:51.129 INFO:teuthology.orchestra.run.smithi070.stdout:2346 trunc from 0xe213e7d to 0x240b97a 2024-09-16T11:43:51.140 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.147 INFO:teuthology.orchestra.run.smithi070.stdout:2348 writesame 0x5107084 thru 0x5107eb1 data_size 0x21(0xe2e bytes) 2024-09-16T11:43:51.157 INFO:teuthology.orchestra.run.smithi070.stdout:2349 write 0x763a430 thru 0x7646e75 (0xca46 bytes) 2024-09-16T11:43:51.163 INFO:teuthology.orchestra.run.smithi070.stdout:2350 trunc from 0x7646e76 to 0x4c104da 2024-09-16T11:43:51.168 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.169 INFO:teuthology.orchestra.run.smithi070.stdout:2352 read 0x3c15580 thru 0x3c1e3b7 (0x8e38 bytes) 2024-09-16T11:43:51.174 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:51.172+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:51.177 INFO:teuthology.orchestra.run.smithi070.stdout:2353 write 0x69e6cd7 thru 0x69f5eac (0xf1d6 bytes) 2024-09-16T11:43:51.180 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.180 INFO:teuthology.orchestra.run.smithi070.stdout:2356 write 0x58001fb thru 0x5807bf7 (0x79fd bytes) 2024-09-16T11:43:51.181 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.181 INFO:teuthology.orchestra.run.smithi070.stdout:2358 read 0x2c7b25d thru 0x2c84922 (0x96c6 bytes) 2024-09-16T11:43:51.182 INFO:teuthology.orchestra.run.smithi070.stdout:2359 punch from 0x5d493ea to 0x5d4ddb8, (0x49ce bytes) 2024-09-16T11:43:51.182 INFO:teuthology.orchestra.run.smithi070.stdout:2361 write 0x31c8739 thru 0x31cd376 (0x4c3e bytes) 2024-09-16T11:43:51.186 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:51.186 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:51.186 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:51.186 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:51.189 INFO:teuthology.orchestra.run.smithi070.stdout:2362 writesame 0x6d03b0a thru 0x6d10241 data_size 0x3c(0xc738 bytes) 2024-09-16T11:43:51.189 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.189 INFO:teuthology.orchestra.run.smithi070.stdout:2364 read 0xc3ff0d thru 0xc44054 (0x4148 bytes) 2024-09-16T11:43:51.190 INFO:teuthology.orchestra.run.smithi070.stdout:2365 read 0x952891 thru 0x95cdfe (0xa56e bytes) 2024-09-16T11:43:51.190 INFO:teuthology.orchestra.run.smithi070.stdout:2366 punch from 0x68a3282 to 0x68b0a98, (0xd816 bytes) 2024-09-16T11:43:51.190 INFO:teuthology.orchestra.run.smithi070.stdout:2367 read 0x157e021 thru 0x1585891 (0x7871 bytes) 2024-09-16T11:43:51.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:43:51.202 INFO:teuthology.orchestra.run.smithi070.stdout:2368 write 0xc4c530a thru 0xc4cdc30 (0x8927 bytes) 2024-09-16T11:43:51.205 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.205 INFO:teuthology.orchestra.run.smithi070.stdout:2370 read 0xa0e37c2 thru 0xa0ecb0a (0x9349 bytes) 2024-09-16T11:43:51.212 INFO:teuthology.orchestra.run.smithi070.stdout:2371 write 0xe2bb44f thru 0xe2c77f4 (0xc3a6 bytes) 2024-09-16T11:43:51.215 INFO:teuthology.orchestra.run.smithi070.stdout:2372 writesame 0x9b062a thru 0x9b12bb data_size 0x1(0xc92 bytes) 2024-09-16T11:43:51.217 INFO:teuthology.orchestra.run.smithi070.stdout:2373 punch from 0xb6f40f5 to 0xb7022e7, (0xe1f2 bytes) 2024-09-16T11:43:51.217 INFO:teuthology.orchestra.run.smithi070.stdout:2374 writesame 0xb8c8734 thru 0xb8cebaa data_size 0x1(0x6477 bytes) 2024-09-16T11:43:51.229 INFO:teuthology.orchestra.run.smithi070.stdout:2375 write 0x415c98c thru 0x416a406 (0xda7b bytes) 2024-09-16T11:43:51.231 INFO:teuthology.orchestra.run.smithi070.stdout:2376 read 0xa2594e2 thru 0xa260707 (0x7226 bytes) 2024-09-16T11:43:51.235 INFO:teuthology.orchestra.run.smithi070.stdout:2377 trunc from 0xe2c77f5 to 0xaaf5ff1 2024-09-16T11:43:51.250 INFO:teuthology.orchestra.run.smithi070.stdout:2378 punch from 0x8284260 to 0x82869a7, (0x2747 bytes) 2024-09-16T11:43:51.251 INFO:teuthology.orchestra.run.smithi070.stdout:2379 trunc from 0xaaf5ff1 to 0xa21fd75 2024-09-16T11:43:51.256 INFO:teuthology.orchestra.run.smithi070.stdout:2380 write 0xa1171c7 thru 0xa11b102 (0x3f3c bytes) 2024-09-16T11:43:51.260 INFO:teuthology.orchestra.run.smithi070.stdout:2381 trunc from 0xa21fd75 to 0xa1bb490 2024-09-16T11:43:51.264 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.264 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.264 INFO:teuthology.orchestra.run.smithi070.stdout:2384 write 0x4b0feb7 thru 0x4b1e770 (0xe8ba bytes) 2024-09-16T11:43:51.267 INFO:teuthology.orchestra.run.smithi070.stdout:2385 write 0xa06a249 thru 0xa077c7e (0xda36 bytes) 2024-09-16T11:43:51.268 INFO:teuthology.orchestra.run.smithi070.stdout:2386 read 0x3809432 thru 0x3818156 (0xed25 bytes) 2024-09-16T11:43:51.268 INFO:teuthology.orchestra.run.smithi070.stdout:2387 writesame 0xe0a04d thru 0xe0c83c data_size 0x47(0x27f0 bytes) 2024-09-16T11:43:51.269 INFO:teuthology.orchestra.run.smithi070.stdout:2388 write 0x46a0009 thru 0x46a5bf6 (0x5bee bytes) 2024-09-16T11:43:51.272 INFO:teuthology.orchestra.run.smithi070.stdout:2389 punch from 0x7e54cba to 0x7e57590, (0x28d6 bytes) 2024-09-16T11:43:51.272 INFO:teuthology.orchestra.run.smithi070.stdout:2390 punch from 0x5992cd0 to 0x5995443, (0x2773 bytes) 2024-09-16T11:43:51.272 INFO:teuthology.orchestra.run.smithi070.stdout:2391 read 0x5e2fcc0 thru 0x5e367ed (0x6b2e bytes) 2024-09-16T11:43:51.272 INFO:teuthology.orchestra.run.smithi070.stdout:2392 punch from 0x67ee9de to 0x67eef63, (0x585 bytes) 2024-09-16T11:43:51.272 INFO:teuthology.orchestra.run.smithi070.stdout:2393 write 0x5bbfa8f thru 0x5bcf92a (0xfe9c bytes) 2024-09-16T11:43:51.274 INFO:teuthology.orchestra.run.smithi070.stdout:2395 write 0x53d4b30 thru 0x53d838e (0x385f bytes) 2024-09-16T11:43:51.274 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:51.272+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:51.275 INFO:teuthology.orchestra.run.smithi070.stdout:2396 write 0x6f9636c thru 0x6f9637d (0x12 bytes) 2024-09-16T11:43:51.276 INFO:teuthology.orchestra.run.smithi070.stdout:2397 write 0x302e2e8 thru 0x3033d56 (0x5a6f bytes) 2024-09-16T11:43:51.278 INFO:teuthology.orchestra.run.smithi070.stdout:2399 punch from 0x7d2d871 to 0x7d3d810, (0xff9f bytes) 2024-09-16T11:43:51.278 INFO:teuthology.orchestra.run.smithi070.stdout:2400 read 0x834c56e thru 0x835000e (0x3aa1 bytes) 2024-09-16T11:43:51.278 INFO:teuthology.orchestra.run.smithi070.stdout:2401 write 0x807acb0 thru 0x8085e59 (0xb1aa bytes) 2024-09-16T11:43:51.279 INFO:teuthology.orchestra.run.smithi070.stdout:2402 read 0x5d6142f thru 0x5d63b00 (0x26d2 bytes) 2024-09-16T11:43:51.280 INFO:teuthology.orchestra.run.smithi070.stdout:2403 trunc from 0xa1bb490 to 0xa4ceb57 2024-09-16T11:43:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:2406 read 0x7ed0835 thru 0x7ed9ce9 (0x94b5 bytes) 2024-09-16T11:43:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:2408 write 0x4de77b8 thru 0x4df298e (0xb1d7 bytes) 2024-09-16T11:43:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:2409 trunc from 0xa4ceb57 to 0xac4d597 2024-09-16T11:43:51.293 INFO:teuthology.orchestra.run.smithi070.stdout:2410 trunc from 0xac4d597 to 0xb3ba953 2024-09-16T11:43:51.308 INFO:teuthology.orchestra.run.smithi070.stdout:2411 trunc from 0xb3ba953 to 0x2a9c280 2024-09-16T11:43:51.308 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:51.308 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:51.308 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:51.309 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:51.309 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:51.318 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:43:51.319 INFO:teuthology.orchestra.run.smithi070.stdout:2412 write 0x1c2514b thru 0x1c2ce49 (0x7cff bytes) 2024-09-16T11:43:51.319 INFO:teuthology.orchestra.run.smithi070.stdout:2413 read 0x11767f9 thru 0x1184a60 (0xe268 bytes) 2024-09-16T11:43:51.338 INFO:teuthology.orchestra.run.smithi070.stdout:2415 writesame 0xeba7e61 thru 0xebac351 data_size 0x9(0x44f1 bytes) 2024-09-16T11:43:51.339 INFO:teuthology.orchestra.run.smithi070.stdout:2416 write 0x724fb6 thru 0x72dff6 (0x9041 bytes) 2024-09-16T11:43:51.340 INFO:teuthology.orchestra.run.smithi070.stdout:2419 read 0xbd70011 thru 0xbd7e0c5 (0xe0b5 bytes) 2024-09-16T11:43:51.340 INFO:teuthology.orchestra.run.smithi070.stdout:2420 write 0xd013c9d thru 0xd014eb2 (0x1216 bytes) 2024-09-16T11:43:51.342 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.342 INFO:teuthology.orchestra.run.smithi070.stdout:2422 punch from 0x80d402a to 0x80da0b3, (0x6089 bytes) 2024-09-16T11:43:51.343 INFO:teuthology.orchestra.run.smithi070.stdout:2424 read 0x73b2133 thru 0x73b3c03 (0x1ad1 bytes) 2024-09-16T11:43:51.343 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.351 INFO:teuthology.orchestra.run.smithi070.stdout:2426 trunc from 0xebac352 to 0x7925c73 2024-09-16T11:43:51.361 INFO:teuthology.orchestra.run.smithi070.stdout:2427 trunc from 0x7925c73 to 0x7a96cbc 2024-09-16T11:43:51.371 INFO:teuthology.orchestra.run.smithi070.stdout:2429 trunc from 0x7a96cbc to 0xbe05d95 2024-09-16T11:43:51.375 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:51.373+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:51.376 INFO:teuthology.orchestra.run.smithi070.stdout:2430 writesame 0x296ae02 thru 0x2976781 data_size 0x70(0xb980 bytes) 2024-09-16T11:43:51.378 INFO:teuthology.orchestra.run.smithi070.stdout:2431 writesame 0x38e6b3d thru 0x38ed9d8 data_size 0x2(0x6e9c bytes) 2024-09-16T11:43:51.386 INFO:teuthology.orchestra.run.smithi070.stdout:2432 writesame 0x1a7485c thru 0x1a7b226 data_size 0x35(0x69cb bytes) 2024-09-16T11:43:51.388 INFO:teuthology.orchestra.run.smithi070.stdout:2433 punch from 0x4ea0f73 to 0x4eaf2af, (0xe33c bytes) 2024-09-16T11:43:51.388 INFO:teuthology.orchestra.run.smithi070.stdout:2434 writesame 0x77f382f thru 0x77f823b data_size 0x3(0x4a0d bytes) 2024-09-16T11:43:51.391 INFO:teuthology.orchestra.run.smithi070.stdout:2435 read 0xaf0152d thru 0xaf08d23 (0x77f7 bytes) 2024-09-16T11:43:51.391 INFO:teuthology.orchestra.run.smithi070.stdout:2437 writesame 0x56d214 thru 0x56d551 data_size 0x1(0x33e bytes) 2024-09-16T11:43:51.392 INFO:teuthology.orchestra.run.smithi070.stdout:2440 punch from 0x3f4c99 to 0x3f6476, (0x17dd bytes) 2024-09-16T11:43:51.392 INFO:teuthology.orchestra.run.smithi070.stdout:2442 read 0x5abd545 thru 0x5ac6de3 (0x989f bytes) 2024-09-16T11:43:51.395 INFO:teuthology.orchestra.run.smithi070.stdout:2443 trunc from 0xbe05d95 to 0x9df9937 2024-09-16T11:43:51.400 INFO:teuthology.orchestra.run.smithi070.stdout:2444 read 0x2a08cf0 thru 0x2a0fb0c (0x6e1d bytes) 2024-09-16T11:43:51.401 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.401 INFO:teuthology.orchestra.run.smithi070.stdout:2448 punch from 0x1718a7d to 0x171a2d1, (0x1854 bytes) 2024-09-16T11:43:51.410 INFO:teuthology.orchestra.run.smithi070.stdout:2449 write 0xd62dc58 thru 0xd63bf3b (0xe2e4 bytes) 2024-09-16T11:43:51.413 INFO:teuthology.orchestra.run.smithi070.stdout:2450 write 0x91207c7 thru 0x912f4d5 (0xed0f bytes) 2024-09-16T11:43:51.414 INFO:teuthology.orchestra.run.smithi070.stdout:2451 read 0x9a9e928 thru 0x9aac13f (0xd818 bytes) 2024-09-16T11:43:51.415 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.415 INFO:teuthology.orchestra.run.smithi070.stdout:2454 write 0x6c61cd0 thru 0x6c6fed3 (0xe204 bytes) 2024-09-16T11:43:51.416 INFO:teuthology.orchestra.run.smithi070.stdout:2456 write 0x5c295ef thru 0x5c2eb78 (0x558a bytes) 2024-09-16T11:43:51.418 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.418 INFO:teuthology.orchestra.run.smithi070.stdout:2459 read 0xcb4e8f6 thru 0xcb5cd38 (0xe443 bytes) 2024-09-16T11:43:51.418 INFO:teuthology.orchestra.run.smithi070.stdout:2461 write 0x85e7a0c thru 0x85f370f (0xbd04 bytes) 2024-09-16T11:43:51.420 INFO:teuthology.orchestra.run.smithi070.stdout:2463 read 0x57b6dbb thru 0x57be25c (0x74a2 bytes) 2024-09-16T11:43:51.420 INFO:teuthology.orchestra.run.smithi070.stdout:2464 punch from 0x95ac460 to 0x95b9f3c, (0xdadc bytes) 2024-09-16T11:43:51.420 INFO:teuthology.orchestra.run.smithi070.stdout:2465 read 0x2c2bef9 thru 0x2c2e774 (0x287c bytes) 2024-09-16T11:43:51.420 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.420 INFO:teuthology.orchestra.run.smithi070.stdout:2467 read 0x8c0d62e thru 0x8c11f1d (0x48f0 bytes) 2024-09-16T11:43:51.421 INFO:teuthology.orchestra.run.smithi070.stdout:2469 writesame 0x1041be7 thru 0x104d086 data_size 0x28(0xb4a0 bytes) 2024-09-16T11:43:51.422 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.427 INFO:teuthology.orchestra.run.smithi070.stdout:2471 trunc from 0xd63bf3c to 0x9545d37 2024-09-16T11:43:51.432 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:51.432 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:51.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:51.432 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:51.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.432 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2367 4.6 4:628f9569:::rbd_data.11c130c94979.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:51.432 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.890791+0000", 2024-09-16T11:43:51.432 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.53832929, 2024-09-16T11:43:51.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0050957820000000001, 2024-09-16T11:43:51.433 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.433 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.433 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.433 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.433 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.433 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2367 2024-09-16T11:43:51.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.433 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.890791+0000", 2024-09-16T11:43:51.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.890791+0000", 2024-09-16T11:43:51.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.890794+0000", 2024-09-16T11:43:51.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3939999999999999e-06 2024-09-16T11:43:51.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891207+0000", 2024-09-16T11:43:51.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00041353000000000001 2024-09-16T11:43:51.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891231+0000", 2024-09-16T11:43:51.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3918999999999999e-05 2024-09-16T11:43:51.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891241+0000", 2024-09-16T11:43:51.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9250000000000007e-06 2024-09-16T11:43:51.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891294+0000", 2024-09-16T11:43:51.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3044999999999998e-05 2024-09-16T11:43:51.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891374+0000", 2024-09-16T11:43:51.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0028e-05 2024-09-16T11:43:51.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:51.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891638+0000", 2024-09-16T11:43:51.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000263486 2024-09-16T11:43:51.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.895515+0000", 2024-09-16T11:43:51.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0038769339999999998 2024-09-16T11:43:51.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.895782+0000", 2024-09-16T11:43:51.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026755600000000001 2024-09-16T11:43:51.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.895841+0000", 2024-09-16T11:43:51.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8736999999999997e-05 2024-09-16T11:43:51.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.895887+0000", 2024-09-16T11:43:51.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6227999999999999e-05 2024-09-16T11:43:51.439 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.439 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.439 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.440 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2368 4.6 4:628f9569:::rbd_data.11c130c94979.0000000000000006:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:51.440 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.891258+0000", 2024-09-16T11:43:51.440 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.537862596, 2024-09-16T11:43:51.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.005177755, 2024-09-16T11:43:51.440 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.440 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.440 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.440 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.440 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.440 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2368 2024-09-16T11:43:51.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.440 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891258+0000", 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891258+0000", 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891259+0000", 2024-09-16T11:43:51.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2699999999999999e-06 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891263+0000", 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.382e-06 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891270+0000", 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4050000000000003e-06 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891275+0000", 2024-09-16T11:43:51.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4429999999999996e-06 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.892982+0000", 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001707241 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.893050+0000", 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.779e-05 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:51.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.893231+0000", 2024-09-16T11:43:51.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0001818 2024-09-16T11:43:51.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.895928+0000", 2024-09-16T11:43:51.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0026962240000000001 2024-09-16T11:43:51.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.896350+0000", 2024-09-16T11:43:51.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00042250099999999999 2024-09-16T11:43:51.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.896394+0000", 2024-09-16T11:43:51.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4045999999999998e-05 2024-09-16T11:43:51.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.896436+0000", 2024-09-16T11:43:51.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1653000000000003e-05 2024-09-16T11:43:51.445 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.446 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.446 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.446 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2472 4.6 4:616b0279:::rbd_data.11c130c94979.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:51.446 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.900159+0000", 2024-09-16T11:43:51.446 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.528961993999999, 2024-09-16T11:43:51.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041559199999999996, 2024-09-16T11:43:51.446 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.446 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.446 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.446 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.446 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.447 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2472 2024-09-16T11:43:51.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.447 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900159+0000", 2024-09-16T11:43:51.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900159+0000", 2024-09-16T11:43:51.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900161+0000", 2024-09-16T11:43:51.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.058e-06 2024-09-16T11:43:51.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900419+0000", 2024-09-16T11:43:51.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0002588 2024-09-16T11:43:51.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900440+0000", 2024-09-16T11:43:51.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0786999999999999e-05 2024-09-16T11:43:51.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900449+0000", 2024-09-16T11:43:51.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7520000000000002e-06 2024-09-16T11:43:51.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900497+0000", 2024-09-16T11:43:51.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7562999999999998e-05 2024-09-16T11:43:51.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900579+0000", 2024-09-16T11:43:51.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2132000000000006e-05 2024-09-16T11:43:51.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:51.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900806+0000", 2024-09-16T11:43:51.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022732699999999999 2024-09-16T11:43:51.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.903786+0000", 2024-09-16T11:43:51.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029796860000000001 2024-09-16T11:43:51.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904243+0000", 2024-09-16T11:43:51.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00045775299999999998 2024-09-16T11:43:51.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904281+0000", 2024-09-16T11:43:51.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7323000000000002e-05 2024-09-16T11:43:51.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904315+0000", 2024-09-16T11:43:51.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3738999999999997e-05 2024-09-16T11:43:51.453 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.453 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2473 4.6 4:616b0279:::rbd_data.11c130c94979.000000000000000c:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.900942+0000", 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.528178369000001, 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037650499999999998, 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2473 2024-09-16T11:43:51.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.455 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900942+0000", 2024-09-16T11:43:51.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900942+0000", 2024-09-16T11:43:51.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900943+0000", 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.099e-06 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900949+0000", 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900962+0000", 2024-09-16T11:43:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.312e-05 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.900969+0000", 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.323e-06 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.902483+0000", 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015145040000000001 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.902575+0000", 2024-09-16T11:43:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1372999999999994e-05 2024-09-16T11:43:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.902749+0000", 2024-09-16T11:43:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000174105 2024-09-16T11:43:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904325+0000", 2024-09-16T11:43:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015760850000000001 2024-09-16T11:43:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904625+0000", 2024-09-16T11:43:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00030017100000000001 2024-09-16T11:43:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904665+0000", 2024-09-16T11:43:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9447999999999999e-05 2024-09-16T11:43:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.904707+0000", 2024-09-16T11:43:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2645000000000003e-05 2024-09-16T11:43:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2784 4.3 4:c63cc7f7:::rbd_data.11c197cf718c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 3940139~50858] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:39.036903+0000", 2024-09-16T11:43:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.392217527, 2024-09-16T11:43:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003760846, 2024-09-16T11:43:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2784 2024-09-16T11:43:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.036903+0000", 2024-09-16T11:43:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.036903+0000", 2024-09-16T11:43:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.036904+0000", 2024-09-16T11:43:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0610000000000001e-06 2024-09-16T11:43:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.037359+0000", 2024-09-16T11:43:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00045501199999999997 2024-09-16T11:43:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.037372+0000", 2024-09-16T11:43:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2683999999999999e-05 2024-09-16T11:43:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.037377+0000", 2024-09-16T11:43:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6949999999999999e-06 2024-09-16T11:43:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.037409+0000", 2024-09-16T11:43:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2378000000000001e-05 2024-09-16T11:43:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.037499+0000", 2024-09-16T11:43:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0000000000000006e-05 2024-09-16T11:43:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.037671+0000", 2024-09-16T11:43:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017164900000000001 2024-09-16T11:43:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.039485+0000", 2024-09-16T11:43:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018147409999999999 2024-09-16T11:43:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.040573+0000", 2024-09-16T11:43:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010876320000000001 2024-09-16T11:43:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.040624+0000", 2024-09-16T11:43:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1332999999999998e-05 2024-09-16T11:43:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:39.040664+0000", 2024-09-16T11:43:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9660999999999999e-05 2024-09-16T11:43:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:4961 4.b 4:d70b03a5:::rbd_data.11c18a2497a7.0000000000000026:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2621440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.755037+0000", 2024-09-16T11:43:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.6740833760000005, 2024-09-16T11:43:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0068672630000000002, 2024-09-16T11:43:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 4961 2024-09-16T11:43:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.755037+0000", 2024-09-16T11:43:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.755037+0000", 2024-09-16T11:43:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.755039+0000", 2024-09-16T11:43:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8759999999999999e-06 2024-09-16T11:43:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.756249+0000", 2024-09-16T11:43:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00121006 2024-09-16T11:43:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.756267+0000", 2024-09-16T11:43:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7785e-05 2024-09-16T11:43:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.756276+0000", 2024-09-16T11:43:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7460000000000006e-06 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.756340+0000", 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4708000000000005e-05 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.756517+0000", 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017614099999999999 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.756805+0000", 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00028886599999999998 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.760673+0000", 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0038680250000000002 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.761845+0000", 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011714570000000001 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.761879+0000", 2024-09-16T11:43:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4233999999999998e-05 2024-09-16T11:43:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.761904+0000", 2024-09-16T11:43:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5364999999999999e-05 2024-09-16T11:43:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5357 4.b 4:d70b03a5:::rbd_data.11c18a2497a7.0000000000000026:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.982429+0000", 2024-09-16T11:43:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.446691457, 2024-09-16T11:43:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.018203652000000001, 2024-09-16T11:43:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5357 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982429+0000", 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982429+0000", 2024-09-16T11:43:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982431+0000", 2024-09-16T11:43:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.705e-06 2024-09-16T11:43:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982438+0000", 2024-09-16T11:43:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.588e-06 2024-09-16T11:43:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982454+0000", 2024-09-16T11:43:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5951e-05 2024-09-16T11:43:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982460+0000", 2024-09-16T11:43:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9329999999999998e-06 2024-09-16T11:43:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982515+0000", 2024-09-16T11:43:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4583000000000002e-05 2024-09-16T11:43:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982609+0000", 2024-09-16T11:43:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3857999999999995e-05 2024-09-16T11:43:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.985720+0000", 2024-09-16T11:43:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031111020000000001 2024-09-16T11:43:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000561+0000", 2024-09-16T11:43:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014841221999999999 2024-09-16T11:43:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000571+0000", 2024-09-16T11:43:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8039999999999996e-06 2024-09-16T11:43:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000606+0000", 2024-09-16T11:43:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5197999999999997e-05 2024-09-16T11:43:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000633+0000", 2024-09-16T11:43:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6707999999999999e-05 2024-09-16T11:43:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.4545.0:5357 4.b e37/28 ondisk, result = 0)", 2024-09-16T11:43:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.995524+0000", 2024-09-16T11:43:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.4335969019999997, 2024-09-16T11:43:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0050393260000000002, 2024-09-16T11:43:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.995524+0000", 2024-09-16T11:43:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.995524+0000", 2024-09-16T11:43:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.995525+0000", 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1489999999999999e-06 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.995529+0000", 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.783e-06 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.995542+0000", 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3202e-05 2024-09-16T11:43:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.995550+0000", 2024-09-16T11:43:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8369999999999997e-06 2024-09-16T11:43:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000549+0000", 2024-09-16T11:43:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0049992769999999999 2024-09-16T11:43:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000556+0000", 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2860000000000002e-06 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000563+0000", 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7920000000000004e-06 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5463 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [stat,truncate 3407872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.143886+0000", 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.2852348669999998, 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0052857260000000001, 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5463 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143886+0000", 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143886+0000", 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143888+0000", 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4889999999999998e-06 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143901+0000", 2024-09-16T11:43:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.291e-05 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143924+0000", 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2523000000000001e-05 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143931+0000", 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144015+0000", 2024-09-16T11:43:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4552e-05 2024-09-16T11:43:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144111+0000", 2024-09-16T11:43:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5310999999999996e-05 2024-09-16T11:43:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144240+0000", 2024-09-16T11:43:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012950900000000001 2024-09-16T11:43:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144947+0000", 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00070658099999999996 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.149102+0000", 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041549810000000003 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.149145+0000", 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3741e-05 2024-09-16T11:43:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.149171+0000", 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6050999999999998e-05 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5488 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [stat,truncate 2621440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.155899+0000", 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.2732211669999991, 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0040559309999999996, 2024-09-16T11:43:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5488 2024-09-16T11:43:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155899+0000", 2024-09-16T11:43:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155899+0000", 2024-09-16T11:43:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155901+0000", 2024-09-16T11:43:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9920000000000002e-06 2024-09-16T11:43:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155912+0000", 2024-09-16T11:43:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0091999999999999e-05 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155933+0000", 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1821999999999999e-05 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155943+0000", 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2560000000000008e-06 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.156006+0000", 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3286000000000004e-05 2024-09-16T11:43:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.156106+0000", 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000100181 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.156252+0000", 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000146434 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.157009+0000", 2024-09-16T11:43:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000756527 2024-09-16T11:43:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.159901+0000", 2024-09-16T11:43:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0028918590000000001 2024-09-16T11:43:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.159932+0000", 2024-09-16T11:43:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0642000000000002e-05 2024-09-16T11:43:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.159955+0000", 2024-09-16T11:43:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3839999999999999e-05 2024-09-16T11:43:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5527 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [stat,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.224549+0000", 2024-09-16T11:43:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.2045714959999998, 2024-09-16T11:43:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046217860000000001, 2024-09-16T11:43:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5527 2024-09-16T11:43:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224549+0000", 2024-09-16T11:43:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224549+0000", 2024-09-16T11:43:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224551+0000", 2024-09-16T11:43:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1100000000000001e-06 2024-09-16T11:43:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224558+0000", 2024-09-16T11:43:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2359999999999998e-06 2024-09-16T11:43:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224581+0000", 2024-09-16T11:43:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2506000000000001e-05 2024-09-16T11:43:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224589+0000", 2024-09-16T11:43:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8839999999999994e-06 2024-09-16T11:43:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224652+0000", 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2791000000000002e-05 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224733+0000", 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1814e-05 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224856+0000", 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0001228 2024-09-16T11:43:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.225410+0000", 2024-09-16T11:43:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00055401500000000004 2024-09-16T11:43:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.229117+0000", 2024-09-16T11:43:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037066109999999998 2024-09-16T11:43:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.229147+0000", 2024-09-16T11:43:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0238000000000001e-05 2024-09-16T11:43:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.229171+0000", 2024-09-16T11:43:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3781000000000002e-05 2024-09-16T11:43:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5998 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.781886+0000", 2024-09-16T11:43:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 8.6472346120000001, 2024-09-16T11:43:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041660059999999999, 2024-09-16T11:43:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5998 2024-09-16T11:43:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781886+0000", 2024-09-16T11:43:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781886+0000", 2024-09-16T11:43:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781887+0000", 2024-09-16T11:43:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3099999999999996e-07 2024-09-16T11:43:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781892+0000", 2024-09-16T11:43:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8690000000000003e-06 2024-09-16T11:43:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781905+0000", 2024-09-16T11:43:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3091999999999999e-05 2024-09-16T11:43:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781911+0000", 2024-09-16T11:43:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.251e-06 2024-09-16T11:43:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781966+0000", 2024-09-16T11:43:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4407999999999998e-05 2024-09-16T11:43:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.782041+0000", 2024-09-16T11:43:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5638999999999999e-05 2024-09-16T11:43:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.782162+0000", 2024-09-16T11:43:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000120659 2024-09-16T11:43:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.782663+0000", 2024-09-16T11:43:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00050085500000000003 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.786004+0000", 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0033415509999999999 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.786030+0000", 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5899000000000001e-05 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.786052+0000", 2024-09-16T11:43:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1852e-05 2024-09-16T11:43:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:6213 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:43.056238+0000", 2024-09-16T11:43:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 8.3728821260000004, 2024-09-16T11:43:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0064010559999999996, 2024-09-16T11:43:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 6213 2024-09-16T11:43:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056238+0000", 2024-09-16T11:43:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056238+0000", 2024-09-16T11:43:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056241+0000", 2024-09-16T11:43:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1679999999999998e-06 2024-09-16T11:43:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056248+0000", 2024-09-16T11:43:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2429999999999999e-06 2024-09-16T11:43:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056267+0000", 2024-09-16T11:43:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9511e-05 2024-09-16T11:43:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056278+0000", 2024-09-16T11:43:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0915e-05 2024-09-16T11:43:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056336+0000", 2024-09-16T11:43:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7349000000000003e-05 2024-09-16T11:43:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056418+0000", 2024-09-16T11:43:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2165999999999999e-05 2024-09-16T11:43:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056538+0000", 2024-09-16T11:43:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012048999999999999 2024-09-16T11:43:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.057059+0000", 2024-09-16T11:43:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00052033099999999996 2024-09-16T11:43:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.062554+0000", 2024-09-16T11:43:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0054957649999999997 2024-09-16T11:43:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.062619+0000", 2024-09-16T11:43:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4329e-05 2024-09-16T11:43:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.062640+0000", 2024-09-16T11:43:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0789e-05 2024-09-16T11:43:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.522 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.522 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.522 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:6552 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:51.522 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:43.445001+0000", 2024-09-16T11:43:51.522 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 7.9841192979999995, 2024-09-16T11:43:51.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037872919999999998, 2024-09-16T11:43:51.522 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.522 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.523 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.523 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.523 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.523 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 6552 2024-09-16T11:43:51.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.523 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.523 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445001+0000", 2024-09-16T11:43:51.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445001+0000", 2024-09-16T11:43:51.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445003+0000", 2024-09-16T11:43:51.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.153e-06 2024-09-16T11:43:51.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445011+0000", 2024-09-16T11:43:51.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6599999999999995e-06 2024-09-16T11:43:51.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.525 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445031+0000", 2024-09-16T11:43:51.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9939e-05 2024-09-16T11:43:51.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.525 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445040+0000", 2024-09-16T11:43:51.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5259999999999993e-06 2024-09-16T11:43:51.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445093+0000", 2024-09-16T11:43:51.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3412999999999998e-05 2024-09-16T11:43:51.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445177+0000", 2024-09-16T11:43:51.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4040000000000002e-05 2024-09-16T11:43:51.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445292+0000", 2024-09-16T11:43:51.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000114737 2024-09-16T11:43:51.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445965+0000", 2024-09-16T11:43:51.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00067304400000000003 2024-09-16T11:43:51.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.528 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.528 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.448739+0000", 2024-09-16T11:43:51.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0027745690000000002 2024-09-16T11:43:51.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.528 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.528 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.448767+0000", 2024-09-16T11:43:51.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7990000000000001e-05 2024-09-16T11:43:51.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.529 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.529 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.448789+0000", 2024-09-16T11:43:51.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1220999999999999e-05 2024-09-16T11:43:51.529 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.529 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.529 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.529 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:6592 4.3 4:c3eff099:::rbd_data.11c18a2497a7.0000000000000000:head [truncate 2621440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:51.529 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:43.473057+0000", 2024-09-16T11:43:51.529 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 7.9560635660000001, 2024-09-16T11:43:51.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0054781910000000003, 2024-09-16T11:43:51.529 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.530 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.530 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.530 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.530 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.530 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 6592 2024-09-16T11:43:51.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.530 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473057+0000", 2024-09-16T11:43:51.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473057+0000", 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473059+0000", 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0660000000000002e-06 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473072+0000", 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2479e-05 2024-09-16T11:43:51.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.532 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473092+0000", 2024-09-16T11:43:51.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0097000000000001e-05 2024-09-16T11:43:51.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.532 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473107+0000", 2024-09-16T11:43:51.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5092999999999999e-05 2024-09-16T11:43:51.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473155+0000", 2024-09-16T11:43:51.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8532000000000002e-05 2024-09-16T11:43:51.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473245+0000", 2024-09-16T11:43:51.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9789000000000004e-05 2024-09-16T11:43:51.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473364+0000", 2024-09-16T11:43:51.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011893200000000001 2024-09-16T11:43:51.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473969+0000", 2024-09-16T11:43:51.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00060456599999999996 2024-09-16T11:43:51.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.478475+0000", 2024-09-16T11:43:51.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004506248 2024-09-16T11:43:51.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.478515+0000", 2024-09-16T11:43:51.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0219999999999998e-05 2024-09-16T11:43:51.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.478535+0000", 2024-09-16T11:43:51.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0169e-05 2024-09-16T11:43:51.535 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.535 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.535 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.536 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:9071 4.3 4:c3279cee:::rbd_data.11c1dc795e15.0000000000000014:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:43:51.536 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.027961+0000", 2024-09-16T11:43:51.536 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 5.4011592329999996, 2024-09-16T11:43:51.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0078018230000000003, 2024-09-16T11:43:51.536 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.536 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.536 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.536 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.536 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.536 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 9071 2024-09-16T11:43:51.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.537 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.027961+0000", 2024-09-16T11:43:51.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.027961+0000", 2024-09-16T11:43:51.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.027962+0000", 2024-09-16T11:43:51.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7899999999999998e-07 2024-09-16T11:43:51.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.027966+0000", 2024-09-16T11:43:51.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2250000000000002e-06 2024-09-16T11:43:51.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.027976+0000", 2024-09-16T11:43:51.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9019999999999994e-06 2024-09-16T11:43:51.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.027980+0000", 2024-09-16T11:43:51.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0330000000000002e-06 2024-09-16T11:43:51.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.028016+0000", 2024-09-16T11:43:51.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5607e-05 2024-09-16T11:43:51.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.028060+0000", 2024-09-16T11:43:51.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4502999999999997e-05 2024-09-16T11:43:51.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.029336+0000", 2024-09-16T11:43:51.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001275782 2024-09-16T11:43:51.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.035687+0000", 2024-09-16T11:43:51.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0063505250000000001 2024-09-16T11:43:51.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.035710+0000", 2024-09-16T11:43:51.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3187999999999999e-05 2024-09-16T11:43:51.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.035740+0000", 2024-09-16T11:43:51.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0046e-05 2024-09-16T11:43:51.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.542 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.035763+0000", 2024-09-16T11:43:51.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3332999999999998e-05 2024-09-16T11:43:51.542 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.542 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.542 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.543 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:9081 4.b 4:d317bc5b:::rbd_data.11c1dc795e15.000000000000000d:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:43:51.543 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:43:51.543 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 5.3751649940000004, 2024-09-16T11:43:51.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027796406999999999, 2024-09-16T11:43:51.543 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.543 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.543 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.543 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.543 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.544 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 9081 2024-09-16T11:43:51.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.544 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:43:51.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:43:51.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053957+0000", 2024-09-16T11:43:51.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7010000000000001e-06 2024-09-16T11:43:51.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053965+0000", 2024-09-16T11:43:51.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5939999999999996e-06 2024-09-16T11:43:51.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053980+0000", 2024-09-16T11:43:51.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5099000000000001e-05 2024-09-16T11:43:51.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053986+0000", 2024-09-16T11:43:51.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5759999999999996e-06 2024-09-16T11:43:51.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.054023+0000", 2024-09-16T11:43:51.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999999e-05 2024-09-16T11:43:51.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.054076+0000", 2024-09-16T11:43:51.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2852999999999997e-05 2024-09-16T11:43:51.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:51.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.058880+0000", 2024-09-16T11:43:51.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048042639999999999 2024-09-16T11:43:51.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081663+0000", 2024-09-16T11:43:51.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022782908000000001 2024-09-16T11:43:51.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081672+0000", 2024-09-16T11:43:51.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0440000000000001e-06 2024-09-16T11:43:51.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081707+0000", 2024-09-16T11:43:51.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4986e-05 2024-09-16T11:43:51.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081752+0000", 2024-09-16T11:43:51.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4781999999999999e-05 2024-09-16T11:43:51.549 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.4545.0:9081 4.b e40/28 ondisk, result = 0)", 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.073300+0000", 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 5.3558206520000002, 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0083654630000000001, 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073300+0000", 2024-09-16T11:43:51.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073300+0000", 2024-09-16T11:43:51.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073301+0000", 2024-09-16T11:43:51.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2100000000000001e-06 2024-09-16T11:43:51.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073307+0000", 2024-09-16T11:43:51.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.071e-06 2024-09-16T11:43:51.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073320+0000", 2024-09-16T11:43:51.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3149000000000001e-05 2024-09-16T11:43:51.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073329+0000", 2024-09-16T11:43:51.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4069999999999993e-06 2024-09-16T11:43:51.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081650+0000", 2024-09-16T11:43:51.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.008321541 2024-09-16T11:43:51.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081658+0000", 2024-09-16T11:43:51.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6260000000000003e-06 2024-09-16T11:43:51.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081665+0000", 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4590000000000001e-06 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:9113 4.b 4:d317bc5b:::rbd_data.11c1dc795e15.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.119551+0000", 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 5.3095695349999996, 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.020116319000000001, 2024-09-16T11:43:51.555 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.556 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.556 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 9113 2024-09-16T11:43:51.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.556 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119551+0000", 2024-09-16T11:43:51.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119551+0000", 2024-09-16T11:43:51.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119553+0000", 2024-09-16T11:43:51.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.782e-06 2024-09-16T11:43:51.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119563+0000", 2024-09-16T11:43:51.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0325e-05 2024-09-16T11:43:51.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119577+0000", 2024-09-16T11:43:51.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.382e-05 2024-09-16T11:43:51.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119583+0000", 2024-09-16T11:43:51.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.57e-06 2024-09-16T11:43:51.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119620+0000", 2024-09-16T11:43:51.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7737000000000001e-05 2024-09-16T11:43:51.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119673+0000", 2024-09-16T11:43:51.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3152000000000002e-05 2024-09-16T11:43:51.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.560 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:51.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.122945+0000", 2024-09-16T11:43:51.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0032719390000000002 2024-09-16T11:43:51.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.560 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139568+0000", 2024-09-16T11:43:51.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016622839 2024-09-16T11:43:51.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.560 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.561 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139606+0000", 2024-09-16T11:43:51.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8006000000000002e-05 2024-09-16T11:43:51.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.561 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139641+0000", 2024-09-16T11:43:51.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5076000000000001e-05 2024-09-16T11:43:51.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139667+0000", 2024-09-16T11:43:51.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6072999999999999e-05 2024-09-16T11:43:51.562 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.562 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.562 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.562 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.4545.0:9113 4.b e40/28 ondisk, result = 0)", 2024-09-16T11:43:51.562 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.133655+0000", 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 5.2954653719999998, 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0060177909999999998, 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.133655+0000", 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.133655+0000", 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.133656+0000", 2024-09-16T11:43:51.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.37e-07 2024-09-16T11:43:51.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.133660+0000", 2024-09-16T11:43:51.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5700000000000001e-06 2024-09-16T11:43:51.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.133672+0000", 2024-09-16T11:43:51.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2201000000000001e-05 2024-09-16T11:43:51.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.133677+0000", 2024-09-16T11:43:51.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3210000000000004e-06 2024-09-16T11:43:51.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139595+0000", 2024-09-16T11:43:51.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0059179929999999999 2024-09-16T11:43:51.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139602+0000", 2024-09-16T11:43:51.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0670000000000003e-06 2024-09-16T11:43:51.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.567 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139673+0000", 2024-09-16T11:43:51.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0902000000000007e-05 2024-09-16T11:43:51.567 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.567 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.567 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.567 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.567 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.567 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:51.568 INFO:teuthology.orchestra.run.smithi070.stdout:2473 punch from 0x8962942 to 0x89646aa, (0x1d68 bytes) 2024-09-16T11:43:51.568 INFO:teuthology.orchestra.run.smithi070.stdout:2475 writesame 0x8a88304 thru 0x8a920b6 data_size 0x1(0x9db3 bytes) 2024-09-16T11:43:51.569 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:43:51.571 INFO:teuthology.orchestra.run.smithi070.stdout:2476 trunc from 0x9545d37 to 0xe52c3c9 2024-09-16T11:43:51.571 INFO:teuthology.orchestra.run.smithi070.stdout:2477 trunc from 0xe52c3c9 to 0x16cbd4d 2024-09-16T11:43:51.571 INFO:teuthology.orchestra.run.smithi070.stdout:2478 write 0x5e355af thru 0x5e357dd (0x22f bytes) 2024-09-16T11:43:51.571 INFO:teuthology.orchestra.run.smithi070.stdout:2479 write 0x2689948 thru 0x26933b5 (0x9a6e bytes) 2024-09-16T11:43:51.571 INFO:teuthology.orchestra.run.smithi070.stdout:2480 trunc from 0x5e357de to 0xaa8c9b6 2024-09-16T11:43:51.572 INFO:teuthology.orchestra.run.smithi070.stdout:2482 write 0x692dcf9 thru 0x6933648 (0x5950 bytes) 2024-09-16T11:43:51.572 INFO:teuthology.orchestra.run.smithi070.stdout:2484 write 0x85f5ffb thru 0x85fce7f (0x6e85 bytes) 2024-09-16T11:43:51.572 INFO:teuthology.orchestra.run.smithi070.stdout:2486 write 0xe574e78 thru 0xe57e644 (0x97cd bytes) 2024-09-16T11:43:51.572 INFO:teuthology.orchestra.run.smithi070.stdout:2487 read 0x9ecfd5d thru 0x9edc87d (0xcb21 bytes) 2024-09-16T11:43:51.572 INFO:teuthology.orchestra.run.smithi070.stdout:2488 read 0xe52be34 thru 0xe53b7e5 (0xf9b2 bytes) 2024-09-16T11:43:51.572 INFO:teuthology.orchestra.run.smithi070.stdout:2489 read 0xcab322c thru 0xcabbaea (0x88bf bytes) 2024-09-16T11:43:51.572 INFO:teuthology.orchestra.run.smithi070.stdout:2490 read 0x9d6683 thru 0x9e5d9c (0xf71a bytes) 2024-09-16T11:43:51.572 INFO:teuthology.orchestra.run.smithi070.stdout:2491 writesame 0xbc9445a thru 0xbc9ce89 data_size 0x21(0x8a30 bytes) 2024-09-16T11:43:51.572 INFO:teuthology.orchestra.run.smithi070.stdout:2492 read 0x60ee96b thru 0x60fb700 (0xcd96 bytes) 2024-09-16T11:43:51.572 INFO:teuthology.orchestra.run.smithi070.stdout:2493 punch from 0xa4abd65 to 0xa4ac570, (0x80b bytes) 2024-09-16T11:43:51.572 INFO:teuthology.orchestra.run.smithi070.stdout:2494 write 0x45f8202 thru 0x45f94bd (0x12bc bytes) 2024-09-16T11:43:51.572 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.573 INFO:teuthology.orchestra.run.smithi070.stdout:2497 write 0x2b6eac5 thru 0x2b777c0 (0x8cfc bytes) 2024-09-16T11:43:51.573 INFO:teuthology.orchestra.run.smithi070.stdout:2498 punch from 0xaaca139 to 0xaad793f, (0xd806 bytes) 2024-09-16T11:43:51.573 INFO:teuthology.orchestra.run.smithi070.stdout:2501 trunc from 0xe57e645 to 0x72c0807 2024-09-16T11:43:51.573 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:51.566+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:51.574 INFO:teuthology.orchestra.run.smithi070.stdout:2503 read 0x3ff3d09 thru 0x3ffe4e9 (0xa7e1 bytes) 2024-09-16T11:43:51.574 INFO:teuthology.orchestra.run.smithi070.stdout:2504 write 0x6a60583 thru 0x6a669c8 (0x6446 bytes) 2024-09-16T11:43:51.582 INFO:teuthology.orchestra.run.smithi070.stdout:2505 writesame 0xc9c9050 thru 0xc9d51a4 data_size 0x1(0xc155 bytes) 2024-09-16T11:43:51.583 INFO:teuthology.orchestra.run.smithi070.stdout:2506 write 0x7f14162 thru 0x7f1afa2 (0x6e41 bytes) 2024-09-16T11:43:51.584 INFO:teuthology.orchestra.run.smithi070.stdout:2507 write 0x8f54140 thru 0x8f5ede8 (0xaca9 bytes) 2024-09-16T11:43:51.588 INFO:teuthology.orchestra.run.smithi070.stdout:2508 trunc from 0xc9d51a5 to 0xec7c765 2024-09-16T11:43:51.596 INFO:teuthology.orchestra.run.smithi070.stdout:2509 trunc from 0xec7c765 to 0xc0a3255 2024-09-16T11:43:51.611 INFO:teuthology.orchestra.run.smithi070.stdout:2510 trunc from 0xc0a3255 to 0x4043479 2024-09-16T11:43:51.624 INFO:teuthology.orchestra.run.smithi070.stdout:2511 write 0x594122a thru 0x594d0f4 (0xbecb bytes) 2024-09-16T11:43:51.629 INFO:teuthology.orchestra.run.smithi070.stdout:2512 trunc from 0x594d0f5 to 0x2167885 2024-09-16T11:43:51.637 INFO:teuthology.orchestra.run.smithi070.stdout:2513 punch from 0x162110f to 0x1625448, (0x4339 bytes) 2024-09-16T11:43:51.638 INFO:teuthology.orchestra.run.smithi070.stdout:2514 write 0x19c4fd5 thru 0x19cc371 (0x739d bytes) 2024-09-16T11:43:51.646 INFO:teuthology.orchestra.run.smithi070.stdout:2515 write 0x487ee6d thru 0x48889e9 (0x9b7d bytes) 2024-09-16T11:43:51.649 INFO:teuthology.orchestra.run.smithi070.stdout:2516 punch from 0x2b214fa to 0x2b2a214, (0x8d1a bytes) 2024-09-16T11:43:51.649 INFO:teuthology.orchestra.run.smithi070.stdout:2518 punch from 0x39d1577 to 0x39d2490, (0xf19 bytes) 2024-09-16T11:43:51.661 INFO:teuthology.orchestra.run.smithi070.stdout:2519 writesame 0xa155e2d thru 0xa158779 data_size 0x1(0x294d bytes) 2024-09-16T11:43:51.661 INFO:teuthology.orchestra.run.smithi070.stdout:2520 writesame 0x8f3d601 thru 0x8f4139a data_size 0x53(0x3d9a bytes) 2024-09-16T11:43:51.663 INFO:teuthology.orchestra.run.smithi070.stdout:2523 write 0x499ff09 thru 0x49ae082 (0xe17a bytes) 2024-09-16T11:43:51.667 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.670 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:51.667+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:51.673 INFO:teuthology.orchestra.run.smithi070.stdout:2525 write 0xc3f6c7c thru 0xc403f98 (0xd31d bytes) 2024-09-16T11:43:51.677 INFO:teuthology.orchestra.run.smithi070.stdout:2526 punch from 0x486f5f5 to 0x4879329, (0x9d34 bytes) 2024-09-16T11:43:51.677 INFO:teuthology.orchestra.run.smithi070.stdout:2527 write 0x5c810b2 thru 0x5c8ce68 (0xbdb7 bytes) 2024-09-16T11:43:51.679 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.680 INFO:teuthology.orchestra.run.smithi070.stdout:2529 trunc from 0xc403f99 to 0xd61c92b 2024-09-16T11:43:51.685 INFO:teuthology.orchestra.run.smithi070.stdout:2530 write 0x3112d5a thru 0x31166c3 (0x396a bytes) 2024-09-16T11:43:51.686 INFO:teuthology.orchestra.run.smithi070.stdout:2531 read 0x5435800 thru 0x5438bb9 (0x33ba bytes) 2024-09-16T11:43:51.686 INFO:teuthology.orchestra.run.smithi070.stdout:2532 write 0x86c3ed0 thru 0x86ce761 (0xa892 bytes) 2024-09-16T11:43:51.687 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:51.687 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:51.687 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:51.687 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:51.688 INFO:teuthology.orchestra.run.smithi070.stdout:2533 read 0x7e7a4ca thru 0x7e7b2f7 (0xe2e bytes) 2024-09-16T11:43:51.688 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.695 INFO:teuthology.orchestra.run.smithi070.stdout:2536 trunc from 0xd61c92b to 0x7ed0b0d 2024-09-16T11:43:51.699 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:43:51.706 INFO:teuthology.orchestra.run.smithi070.stdout:2537 write 0xed31ad thru 0xedb83d (0x8691 bytes) 2024-09-16T11:43:51.707 INFO:teuthology.orchestra.run.smithi070.stdout:2538 write 0x15fbe36 thru 0x160771c (0xb8e7 bytes) 2024-09-16T11:43:51.708 INFO:teuthology.orchestra.run.smithi070.stdout:2539 read 0x17c5ba thru 0x17e731 (0x2178 bytes) 2024-09-16T11:43:51.709 INFO:teuthology.orchestra.run.smithi070.stdout:2540 punch from 0x9763c8 to 0x97b4eb, (0x5123 bytes) 2024-09-16T11:43:51.709 INFO:teuthology.orchestra.run.smithi070.stdout:2542 punch from 0x2755a96 to 0x2759ae1, (0x404b bytes) 2024-09-16T11:43:51.709 INFO:teuthology.orchestra.run.smithi070.stdout:2543 write 0x1ffc666 thru 0x200be4a (0xf7e5 bytes) 2024-09-16T11:43:51.710 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.710 INFO:teuthology.orchestra.run.smithi070.stdout:2546 read 0x3ee3e72 thru 0x3eef3f6 (0xb585 bytes) 2024-09-16T11:43:51.710 INFO:teuthology.orchestra.run.smithi070.stdout:2548 read 0x115d13e thru 0x1165b45 (0x8a08 bytes) 2024-09-16T11:43:51.711 INFO:teuthology.orchestra.run.smithi070.stdout:2549 punch from 0x78eb126 to 0x78ed014, (0x1eee bytes) 2024-09-16T11:43:51.724 INFO:teuthology.orchestra.run.smithi070.stdout:2552 write 0xec4e48a thru 0xec57f7b (0x9af2 bytes) 2024-09-16T11:43:51.726 INFO:teuthology.orchestra.run.smithi070.stdout:2554 write 0xad196e9 thru 0xad232d6 (0x9bee bytes) 2024-09-16T11:43:51.729 INFO:teuthology.orchestra.run.smithi070.stdout:2555 read 0x61de89 thru 0x62c29e (0xe416 bytes) 2024-09-16T11:43:51.729 INFO:teuthology.orchestra.run.smithi070.stdout:2559 writesame 0xc4d64e8 thru 0xc4e3ee3 data_size 0xe(0xd9fc bytes) 2024-09-16T11:43:51.732 INFO:teuthology.orchestra.run.smithi070.stdout:2560 writesame 0x2cf3b36 thru 0x2d009b8 data_size 0x1(0xce83 bytes) 2024-09-16T11:43:51.733 INFO:teuthology.orchestra.run.smithi070.stdout:2562 trunc from 0xec57f7c to 0xe8cd803 2024-09-16T11:43:51.747 INFO:teuthology.orchestra.run.smithi070.stdout:2564 trunc from 0xe8cd803 to 0x8007c23 2024-09-16T11:43:51.761 INFO:teuthology.orchestra.run.smithi070.stdout:2566 write 0x4a0ba6a thru 0x4a0e0ec (0x2683 bytes) 2024-09-16T11:43:51.763 INFO:teuthology.orchestra.run.smithi070.stdout:2568 punch from 0x31a7fa1 to 0x31a9269, (0x12c8 bytes) 2024-09-16T11:43:51.763 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.763 INFO:teuthology.orchestra.run.smithi070.stdout:2571 write 0x5efc30a thru 0x5f03589 (0x7280 bytes) 2024-09-16T11:43:51.764 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.764 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.770 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:51.769+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:51.773 INFO:teuthology.orchestra.run.smithi070.stdout:2574 write 0xbb5f387 thru 0xbb6bdb7 (0xca31 bytes) 2024-09-16T11:43:51.776 INFO:teuthology.orchestra.run.smithi070.stdout:2575 write 0x42a06ae thru 0x42af81f (0xf172 bytes) 2024-09-16T11:43:51.778 INFO:teuthology.orchestra.run.smithi070.stdout:2576 read 0x781f12b thru 0x782b5c0 (0xc496 bytes) 2024-09-16T11:43:51.779 INFO:teuthology.orchestra.run.smithi070.stdout:2577 read 0x136d167 thru 0x13757de (0x8678 bytes) 2024-09-16T11:43:51.779 INFO:teuthology.orchestra.run.smithi070.stdout:2578 read 0xf9f527 thru 0xfa977a (0xa254 bytes) 2024-09-16T11:43:51.779 INFO:teuthology.orchestra.run.smithi070.stdout:2579 write 0x5349143 thru 0x534f479 (0x6337 bytes) 2024-09-16T11:43:51.781 INFO:teuthology.orchestra.run.smithi070.stdout:2580 write 0x49f6fcb thru 0x49fae19 (0x3e4f bytes) 2024-09-16T11:43:51.782 INFO:teuthology.orchestra.run.smithi070.stdout:2581 writesame 0x1ee2be9 thru 0x1ef1cdf data_size 0x1(0xf0f7 bytes) 2024-09-16T11:43:51.809 INFO:teuthology.orchestra.run.smithi070.stdout:2582 read 0x2adeb21 thru 0x2ae5796 (0x6c76 bytes) 2024-09-16T11:43:51.810 INFO:teuthology.orchestra.run.smithi070.stdout:2583 punch from 0xa4630c0 to 0xa465b4d, (0x2a8d bytes) 2024-09-16T11:43:51.810 INFO:teuthology.orchestra.run.smithi070.stdout:2584 writesame 0x294651c thru 0x294bbc3 data_size 0x5e(0x56a8 bytes) 2024-09-16T11:43:51.814 INFO:teuthology.orchestra.run.smithi070.stdout:2585 trunc from 0xbb6bdb8 to 0x99e1c8c 2024-09-16T11:43:51.815 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:51.815 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:51.815 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:51.815 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:51.815 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:51.824 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:43:51.824 INFO:teuthology.orchestra.run.smithi070.stdout:2587 read 0x5bc8ee6 thru 0x5bd6128 (0xd243 bytes) 2024-09-16T11:43:51.825 INFO:teuthology.orchestra.run.smithi070.stdout:2589 writesame 0x7d82e12 thru 0x7d83510 data_size 0x3(0x6ff bytes) 2024-09-16T11:43:51.837 INFO:teuthology.orchestra.run.smithi070.stdout:2592 write 0xca431a5 thru 0xca4d32d (0xa189 bytes) 2024-09-16T11:43:51.841 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T11:43:51.844 INFO:teuthology.orchestra.run.smithi070.stdout:2593 read 0x8349a99 thru 0x834d121 (0x3689 bytes) 2024-09-16T11:43:51.845 INFO:teuthology.orchestra.run.smithi070.stdout:2594 punch from 0x776c49d to 0x777476c, (0x82cf bytes) 2024-09-16T11:43:51.851 INFO:teuthology.orchestra.run.smithi070.stdout:2595 trunc from 0xca4d32e to 0x755a437 2024-09-16T11:43:51.866 INFO:teuthology.orchestra.run.smithi070.stdout:2596 read 0x46340a2 thru 0x4642f58 (0xeeb7 bytes) 2024-09-16T11:43:51.872 INFO:teuthology.orchestra.run.smithi070.stdout:2597 trunc from 0x755a437 to 0xbbbecfd 2024-09-16T11:43:51.873 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:51.871+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:51.877 INFO:teuthology.orchestra.run.smithi070.stdout:2598 write 0x8a4491 thru 0x8a8f60 (0x4ad0 bytes) 2024-09-16T11:43:51.878 INFO:teuthology.orchestra.run.smithi070.stdout:2599 punch from 0x6d4e386 to 0x6d52d6d, (0x49e7 bytes) 2024-09-16T11:43:51.879 INFO:teuthology.orchestra.run.smithi070.stdout:2601 write 0xa8b009e thru 0xa8b564c (0x55af bytes) 2024-09-16T11:43:51.881 INFO:teuthology.orchestra.run.smithi070.stdout:2602 read 0x35ebe47 thru 0x35ed3d2 (0x158c bytes) 2024-09-16T11:43:51.882 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.882 INFO:teuthology.orchestra.run.smithi070.stdout:2605 writesame 0xb6c5a06 thru 0xb6d5128 data_size 0x1(0xf723 bytes) 2024-09-16T11:43:51.916 INFO:teuthology.orchestra.run.smithi070.stdout:2606 trunc from 0xbbbecfd to 0xa6bf389 2024-09-16T11:43:51.925 INFO:teuthology.orchestra.run.smithi070.stdout:2607 punch from 0x371816f to 0x37196e6, (0x1577 bytes) 2024-09-16T11:43:51.926 INFO:teuthology.orchestra.run.smithi070.stdout:2610 writesame 0x238f580 thru 0x239592e data_size 0xd(0x63af bytes) 2024-09-16T11:43:51.927 INFO:teuthology.orchestra.run.smithi070.stdout:2611 writesame 0x84f77a4 thru 0x85003ef data_size 0x7b(0x8c4c bytes) 2024-09-16T11:43:51.928 INFO:teuthology.orchestra.run.smithi070.stdout:2613 read 0x667fb9b thru 0x668190a (0x1d70 bytes) 2024-09-16T11:43:51.929 INFO:teuthology.orchestra.run.smithi070.stdout:2614 write 0x37dea07 thru 0x37eb424 (0xca1e bytes) 2024-09-16T11:43:51.939 INFO:teuthology.orchestra.run.smithi070.stdout:2616 writesame 0xcfa8b62 thru 0xcfaaf97 data_size 0x1e(0x2436 bytes) 2024-09-16T11:43:51.939 INFO:teuthology.orchestra.run.smithi070.stdout:2617 read 0xc653292 thru 0xc662424 (0xf193 bytes) 2024-09-16T11:43:51.939 INFO:teuthology.orchestra.run.smithi070.stdout:2619 punch from 0x53430ea to 0x5343fdc, (0xef2 bytes) 2024-09-16T11:43:51.939 INFO:teuthology.orchestra.run.smithi070.stdout:2621 punch from 0x688e1c3 to 0x689654b, (0x8388 bytes) 2024-09-16T11:43:51.939 INFO:teuthology.orchestra.run.smithi070.stdout:2622 punch from 0xd4aa05 to 0xd4e848, (0x3e43 bytes) 2024-09-16T11:43:51.939 INFO:teuthology.orchestra.run.smithi070.stdout:2624 read 0x78057a5 thru 0x781162e (0xbe8a bytes) 2024-09-16T11:43:51.939 INFO:teuthology.orchestra.run.smithi070.stdout:2625 write 0x6f61f96 thru 0x6f6dc5a (0xbcc5 bytes) 2024-09-16T11:43:51.940 INFO:teuthology.orchestra.run.smithi070.stdout:2626 read 0xb2dd22c thru 0xb2e1455 (0x422a bytes) 2024-09-16T11:43:51.941 INFO:teuthology.orchestra.run.smithi070.stdout:2629 read 0x410c1d5 thru 0x411670e (0xa53a bytes) 2024-09-16T11:43:51.941 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.941 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.941 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:51.941 INFO:teuthology.orchestra.run.smithi070.stdout:2637 read 0x6f49d02 thru 0x6f56ec8 (0xd1c7 bytes) 2024-09-16T11:43:51.942 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:51.942 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:51.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:51.942 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:51.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.943 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1159 4.7 4:e7dc364e:::rbd_data.11c17b8dc4e5.0000000000000004:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:51.943 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.349936+0000", 2024-09-16T11:43:51.943 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.58853073, 2024-09-16T11:43:51.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.017071449999999998, 2024-09-16T11:43:51.943 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.943 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.943 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.943 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.943 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.943 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1159 2024-09-16T11:43:51.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.943 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.349936+0000", 2024-09-16T11:43:51.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.349936+0000", 2024-09-16T11:43:51.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.349937+0000", 2024-09-16T11:43:51.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.004e-06 2024-09-16T11:43:51.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.349942+0000", 2024-09-16T11:43:51.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2290000000000002e-06 2024-09-16T11:43:51.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.349956+0000", 2024-09-16T11:43:51.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3668e-05 2024-09-16T11:43:51.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.349962+0000", 2024-09-16T11:43:51.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:43:51.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.350001+0000", 2024-09-16T11:43:51.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8868000000000001e-05 2024-09-16T11:43:51.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.350048+0000", 2024-09-16T11:43:51.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6860000000000002e-05 2024-09-16T11:43:51.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.352855+0000", 2024-09-16T11:43:51.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002807047 2024-09-16T11:43:51.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.366886+0000", 2024-09-16T11:43:51.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014030519 2024-09-16T11:43:51.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.366897+0000", 2024-09-16T11:43:51.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1813e-05 2024-09-16T11:43:51.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.366948+0000", 2024-09-16T11:43:51.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0932e-05 2024-09-16T11:43:51.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.367007+0000", 2024-09-16T11:43:51.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9120999999999999e-05 2024-09-16T11:43:51.948 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.948 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.949 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.949 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.4545.0:1159 4.7 e32/28 ondisk, result = 0)", 2024-09-16T11:43:51.949 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.362874+0000", 2024-09-16T11:43:51.949 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.575592603, 2024-09-16T11:43:51.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0040141250000000003, 2024-09-16T11:43:51.949 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.949 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:51.949 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.362874+0000", 2024-09-16T11:43:51.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.362874+0000", 2024-09-16T11:43:51.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.362875+0000", 2024-09-16T11:43:51.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8599999999999996e-07 2024-09-16T11:43:51.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.362880+0000", 2024-09-16T11:43:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3959999999999997e-06 2024-09-16T11:43:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.362894+0000", 2024-09-16T11:43:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3796e-05 2024-09-16T11:43:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.362902+0000", 2024-09-16T11:43:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3420000000000002e-06 2024-09-16T11:43:51.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.952 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.952 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.366868+0000", 2024-09-16T11:43:51.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003966748 2024-09-16T11:43:51.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.952 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.952 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.366878+0000", 2024-09-16T11:43:51.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6460000000000005e-06 2024-09-16T11:43:51.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.952 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.953 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.366888+0000", 2024-09-16T11:43:51.953 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0210999999999999e-05 2024-09-16T11:43:51.953 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.953 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.953 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.953 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1164 4.7 4:e7dc364e:::rbd_data.11c17b8dc4e5.0000000000000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:51.953 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.381830+0000", 2024-09-16T11:43:51.953 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.556636710999999, 2024-09-16T11:43:51.953 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037668049999999998, 2024-09-16T11:43:51.953 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.953 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.954 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.954 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.954 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.954 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1164 2024-09-16T11:43:51.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.954 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381830+0000", 2024-09-16T11:43:51.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381830+0000", 2024-09-16T11:43:51.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381833+0000", 2024-09-16T11:43:51.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.819e-06 2024-09-16T11:43:51.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381843+0000", 2024-09-16T11:43:51.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.041e-05 2024-09-16T11:43:51.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381864+0000", 2024-09-16T11:43:51.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0441999999999998e-05 2024-09-16T11:43:51.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381872+0000", 2024-09-16T11:43:51.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2919999999999996e-06 2024-09-16T11:43:51.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.957 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.957 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381928+0000", 2024-09-16T11:43:51.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5800000000000001e-05 2024-09-16T11:43:51.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.957 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.957 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382025+0000", 2024-09-16T11:43:51.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6965000000000004e-05 2024-09-16T11:43:51.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.957 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.957 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382156+0000", 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000131268 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382936+0000", 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00077983600000000001 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.385538+0000", 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0026021949999999999 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.385563+0000", 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4851e-05 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.385597+0000", 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3927000000000002e-05 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1169 4.7 4:e4d3be82:::rbd_data.11c17b8dc4e5.0000000000000019:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:51.959 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.381901+0000", 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.556565752999999, 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0067125639999999999, 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1169 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381901+0000", 2024-09-16T11:43:51.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381901+0000", 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381901+0000", 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3000000000000002e-07 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381903+0000", 2024-09-16T11:43:51.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1390000000000002e-06 2024-09-16T11:43:51.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381908+0000", 2024-09-16T11:43:51.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9089999999999999e-06 2024-09-16T11:43:51.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381912+0000", 2024-09-16T11:43:51.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:43:51.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382434+0000", 2024-09-16T11:43:51.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00052215200000000001 2024-09-16T11:43:51.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382504+0000", 2024-09-16T11:43:51.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9612999999999999e-05 2024-09-16T11:43:51.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382608+0000", 2024-09-16T11:43:51.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103961 2024-09-16T11:43:51.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.383230+0000", 2024-09-16T11:43:51.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00062237600000000001 2024-09-16T11:43:51.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.388527+0000", 2024-09-16T11:43:51.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.005296991 2024-09-16T11:43:51.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.388574+0000", 2024-09-16T11:43:51.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7005999999999997e-05 2024-09-16T11:43:51.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.388614+0000", 2024-09-16T11:43:51.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9521000000000003e-05 2024-09-16T11:43:51.965 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.965 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.965 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.966 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1170 4.2 4:4447a4c2:::rbd_data.11c17b8dc4e5.000000000000001d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-16T11:43:51.966 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:37.381916+0000", 2024-09-16T11:43:51.966 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.556550883, 2024-09-16T11:43:51.966 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0036735909999999999, 2024-09-16T11:43:51.966 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.966 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.966 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.966 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.966 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.966 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1170 2024-09-16T11:43:51.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381916+0000", 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381916+0000", 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381916+0000", 2024-09-16T11:43:51.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2800000000000003e-07 2024-09-16T11:43:51.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381919+0000", 2024-09-16T11:43:51.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.106e-06 2024-09-16T11:43:51.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381925+0000", 2024-09-16T11:43:51.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:43:51.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381931+0000", 2024-09-16T11:43:51.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6869999999999997e-06 2024-09-16T11:43:51.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.381968+0000", 2024-09-16T11:43:51.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6631000000000002e-05 2024-09-16T11:43:51.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382015+0000", 2024-09-16T11:43:51.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7237999999999997e-05 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382087+0000", 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1958000000000005e-05 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.382888+0000", 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00080099999999999995 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.385520+0000", 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0026317379999999998 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.385549+0000", 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9345000000000001e-05 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:37.385589+0000", 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0003000000000003e-05 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.971 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.972 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1605 4.7 4:e61afa3b:::rbd_data.11c130c94979.00000000000000de:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T11:43:51.972 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.323019+0000", 2024-09-16T11:43:51.972 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 13.615447662999999, 2024-09-16T11:43:51.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046077990000000001, 2024-09-16T11:43:51.972 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.972 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.972 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.972 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.972 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.972 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1605 2024-09-16T11:43:51.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.973 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.973 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.323019+0000", 2024-09-16T11:43:51.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.973 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.323019+0000", 2024-09-16T11:43:51.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.323020+0000", 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3600000000000002e-07 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.323024+0000", 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2490000000000003e-06 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.323035+0000", 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0783e-05 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.323040+0000", 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2989999999999996e-06 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.323082+0000", 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1779000000000001e-05 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.323162+0000", 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0328999999999996e-05 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.323257+0000", 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4417000000000001e-05 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.323676+0000", 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00041903699999999998 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.327580+0000", 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039041340000000001 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.327607+0000", 2024-09-16T11:43:51.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7308999999999999e-05 2024-09-16T11:43:51.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.327627+0000", 2024-09-16T11:43:51.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9627e-05 2024-09-16T11:43:51.977 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.977 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.977 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.977 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4545.0:4954 4.14 e37/28 4:2b6429f2:::rbd_data.11c18a2497a7.0000000000000027:head v 37'18, mlcod=35'16)", 2024-09-16T11:43:51.978 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.752593+0000", 2024-09-16T11:43:51.978 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.185873412999999, 2024-09-16T11:43:51.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0049490020000000001, 2024-09-16T11:43:51.978 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.978 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.978 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.752593+0000", 2024-09-16T11:43:51.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.752593+0000", 2024-09-16T11:43:51.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.752594+0000", 2024-09-16T11:43:51.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8800000000000001e-07 2024-09-16T11:43:51.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.754163+0000", 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001568727 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.754175+0000", 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1687e-05 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.754181+0000", 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3119999999999999e-06 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.754236+0000", 2024-09-16T11:43:51.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5427000000000003e-05 2024-09-16T11:43:51.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.754259+0000", 2024-09-16T11:43:51.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2464e-05 2024-09-16T11:43:51.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.757481+0000", 2024-09-16T11:43:51.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003222286 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.757542+0000", 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1210999999999994e-05 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4545.0:4961 4.b e37/28 4:d70b03a5:::rbd_data.11c18a2497a7.0000000000000026:head v 37'11, mlcod=36'9)", 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.757016+0000", 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.181451150000001, 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047034620000000003, 2024-09-16T11:43:51.982 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.757016+0000", 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.757016+0000", 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.757017+0000", 2024-09-16T11:43:51.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2240000000000001e-06 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.758588+0000", 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015711099999999999 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.758602+0000", 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3791999999999999e-05 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.758608+0000", 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5950000000000004e-06 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.758657+0000", 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9212999999999998e-05 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.758682+0000", 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4652999999999999e-05 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.761667+0000", 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029845200000000001 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.761719+0000", 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2355000000000003e-05 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:4981 4.2 4:44657293:::rbd_data.11c18a2497a7.0000000000000025:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:51.986 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.775880+0000", 2024-09-16T11:43:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.162586488000001, 2024-09-16T11:43:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0034994929999999998, 2024-09-16T11:43:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 4981 2024-09-16T11:43:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.775880+0000", 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.775880+0000", 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.775881+0000", 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.75e-07 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.775883+0000", 2024-09-16T11:43:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6690000000000002e-06 2024-09-16T11:43:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.775892+0000", 2024-09-16T11:43:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7350000000000002e-06 2024-09-16T11:43:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.775897+0000", 2024-09-16T11:43:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7180000000000004e-06 2024-09-16T11:43:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.775942+0000", 2024-09-16T11:43:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5352999999999998e-05 2024-09-16T11:43:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.776016+0000", 2024-09-16T11:43:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3236000000000002e-05 2024-09-16T11:43:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.776119+0000", 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103207 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.776836+0000", 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00071687900000000002 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.779338+0000", 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025020210000000001 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.779362+0000", 2024-09-16T11:43:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4803999999999998e-05 2024-09-16T11:43:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.779380+0000", 2024-09-16T11:43:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7295999999999998e-05 2024-09-16T11:43:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5473 4.12 4:48b6ec57:::rbd_data.11c18a2497a7.000000000000000a:head [stat,truncate 3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.143965+0000", 2024-09-16T11:43:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.7945016280000008, 2024-09-16T11:43:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0052763269999999999, 2024-09-16T11:43:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5473 2024-09-16T11:43:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143965+0000", 2024-09-16T11:43:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143965+0000", 2024-09-16T11:43:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143967+0000", 2024-09-16T11:43:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0940000000000002e-06 2024-09-16T11:43:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143974+0000", 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8299999999999998e-06 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143991+0000", 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6772000000000001e-05 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143998+0000", 2024-09-16T11:43:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.481e-06 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144051+0000", 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2618999999999999e-05 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144133+0000", 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144261+0000", 2024-09-16T11:43:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000128462 2024-09-16T11:43:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.145084+0000", 2024-09-16T11:43:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00082212599999999995 2024-09-16T11:43:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.149162+0000", 2024-09-16T11:43:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0040787130000000003 2024-09-16T11:43:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.149206+0000", 2024-09-16T11:43:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4242000000000001e-05 2024-09-16T11:43:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.149241+0000", 2024-09-16T11:43:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4915e-05 2024-09-16T11:43:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5537 4.12 4:48b6ec57:::rbd_data.11c18a2497a7.000000000000000a:head [stat,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.224646+0000", 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.7138206369999995, 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0045771509999999998, 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5537 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224646+0000", 2024-09-16T11:43:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224646+0000", 2024-09-16T11:43:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224648+0000", 2024-09-16T11:43:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0099999999999998e-06 2024-09-16T11:43:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224654+0000", 2024-09-16T11:43:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3790000000000003e-06 2024-09-16T11:43:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224674+0000", 2024-09-16T11:43:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9403e-05 2024-09-16T11:43:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224682+0000", 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0129999999999993e-06 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224745+0000", 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2864e-05 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224828+0000", 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3416000000000002e-05 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.224959+0000", 2024-09-16T11:43:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000130445 2024-09-16T11:43:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.225769+0000", 2024-09-16T11:43:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00081033199999999998 2024-09-16T11:43:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.229145+0000", 2024-09-16T11:43:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0033760330000000001 2024-09-16T11:43:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.229191+0000", 2024-09-16T11:43:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6187999999999999e-05 2024-09-16T11:43:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.229223+0000", 2024-09-16T11:43:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2067999999999999e-05 2024-09-16T11:43:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5664 4.12 4:48b6ec57:::rbd_data.11c18a2497a7.000000000000000a:head [truncate 3670016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.361879+0000", 2024-09-16T11:43:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.5765878240000006, 2024-09-16T11:43:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0035585429999999999, 2024-09-16T11:43:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5664 2024-09-16T11:43:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.361879+0000", 2024-09-16T11:43:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.361879+0000", 2024-09-16T11:43:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.361881+0000", 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.361886+0000", 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.339e-06 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.361902+0000", 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5841999999999998e-05 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.361908+0000", 2024-09-16T11:43:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5379999999999999e-06 2024-09-16T11:43:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.361952+0000", 2024-09-16T11:43:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3720999999999997e-05 2024-09-16T11:43:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.362034+0000", 2024-09-16T11:43:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1988000000000002e-05 2024-09-16T11:43:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.362158+0000", 2024-09-16T11:43:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000123953 2024-09-16T11:43:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.362781+0000", 2024-09-16T11:43:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00062275899999999996 2024-09-16T11:43:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.365370+0000", 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025889620000000002 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.365405+0000", 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5330999999999998e-05 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.365437+0000", 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2276000000000001e-05 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:6008 4.12 4:48b6ec57:::rbd_data.11c18a2497a7.000000000000000a:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.781970+0000", 2024-09-16T11:43:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.1564968049999997, 2024-09-16T11:43:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0043645120000000001, 2024-09-16T11:43:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 6008 2024-09-16T11:43:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781970+0000", 2024-09-16T11:43:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781970+0000", 2024-09-16T11:43:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781971+0000", 2024-09-16T11:43:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4789999999999999e-06 2024-09-16T11:43:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.781977+0000", 2024-09-16T11:43:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6030000000000004e-06 2024-09-16T11:43:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.782003+0000", 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5607000000000001e-05 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.782021+0000", 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8797e-05 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.782092+0000", 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0838000000000002e-05 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.782172+0000", 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9671000000000003e-05 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.782285+0000", 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011349399999999999 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.782667+0000", 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00038155699999999998 2024-09-16T11:43:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.786283+0000", 2024-09-16T11:43:52.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003616232 2024-09-16T11:43:52.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.786313+0000", 2024-09-16T11:43:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9822e-05 2024-09-16T11:43:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.786334+0000", 2024-09-16T11:43:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1412000000000001e-05 2024-09-16T11:43:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:6159 4.12 4:48b6ec57:::rbd_data.11c18a2497a7.000000000000000a:head [truncate 1310720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.950796+0000", 2024-09-16T11:43:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 8.9876711300000007, 2024-09-16T11:43:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0060023189999999999, 2024-09-16T11:43:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 6159 2024-09-16T11:43:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.950796+0000", 2024-09-16T11:43:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.950796+0000", 2024-09-16T11:43:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.950796+0000", 2024-09-16T11:43:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0299999999999998e-07 2024-09-16T11:43:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.950800+0000", 2024-09-16T11:43:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9550000000000002e-06 2024-09-16T11:43:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.950809+0000", 2024-09-16T11:43:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1519999999999998e-06 2024-09-16T11:43:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.950814+0000", 2024-09-16T11:43:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7829999999999998e-06 2024-09-16T11:43:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.950855+0000", 2024-09-16T11:43:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0850999999999999e-05 2024-09-16T11:43:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.950923+0000", 2024-09-16T11:43:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7471999999999994e-05 2024-09-16T11:43:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.951045+0000", 2024-09-16T11:43:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000122002 2024-09-16T11:43:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.951790+0000", 2024-09-16T11:43:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00074508699999999996 2024-09-16T11:43:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.956751+0000", 2024-09-16T11:43:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004961521 2024-09-16T11:43:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.956779+0000", 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7838999999999999e-05 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.956798+0000", 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8853999999999999e-05 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:6223 4.12 4:48b6ec57:::rbd_data.11c18a2497a7.000000000000000a:head [truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:43.056311+0000", 2024-09-16T11:43:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 8.8821556820000005, 2024-09-16T11:43:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0034644390000000001, 2024-09-16T11:43:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 6223 2024-09-16T11:43:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056311+0000", 2024-09-16T11:43:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056311+0000", 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056313+0000", 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.657e-06 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056319+0000", 2024-09-16T11:43:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9490000000000001e-06 2024-09-16T11:43:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056333+0000", 2024-09-16T11:43:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4819e-05 2024-09-16T11:43:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056340+0000", 2024-09-16T11:43:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2829999999999998e-06 2024-09-16T11:43:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056390+0000", 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0244999999999998e-05 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056447+0000", 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6987999999999997e-05 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056541+0000", 2024-09-16T11:43:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3937000000000005e-05 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.056922+0000", 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00038136000000000002 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.059727+0000", 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0028042539999999999 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.059755+0000", 2024-09-16T11:43:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8340999999999999e-05 2024-09-16T11:43:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.059775+0000", 2024-09-16T11:43:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0605999999999999e-05 2024-09-16T11:43:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:6562 4.12 4:48b6ec57:::rbd_data.11c18a2497a7.000000000000000a:head [truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:43.445006+0000", 2024-09-16T11:43:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 8.4934609299999995, 2024-09-16T11:43:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003737228, 2024-09-16T11:43:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 6562 2024-09-16T11:43:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445006+0000", 2024-09-16T11:43:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445006+0000", 2024-09-16T11:43:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445007+0000", 2024-09-16T11:43:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5300000000000003e-07 2024-09-16T11:43:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445011+0000", 2024-09-16T11:43:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0770000000000001e-06 2024-09-16T11:43:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445021+0000", 2024-09-16T11:43:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0165e-05 2024-09-16T11:43:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445027+0000", 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445091+0000", 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3943999999999996e-05 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445163+0000", 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2601000000000005e-05 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445268+0000", 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104978 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.445990+0000", 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00072170399999999999 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.448688+0000", 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0026978499999999999 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.448718+0000", 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0018999999999998e-05 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.448743+0000", 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5247000000000001e-05 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:6602 4.12 4:48b6ec57:::rbd_data.11c18a2497a7.000000000000000a:head [truncate 2586791] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:43.473160+0000", 2024-09-16T11:43:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 8.4653064279999999, 2024-09-16T11:43:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0053148509999999998, 2024-09-16T11:43:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 6602 2024-09-16T11:43:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473160+0000", 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473160+0000", 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473161+0000", 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4499999999999996e-07 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473166+0000", 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0039999999999999e-06 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473180+0000", 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3636e-05 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473186+0000", 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1109999999999996e-06 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473234+0000", 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8615000000000001e-05 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473303+0000", 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8700000000000003e-05 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 6", 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.473402+0000", 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8546000000000001e-05 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.474027+0000", 2024-09-16T11:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00062522599999999999 2024-09-16T11:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.478425+0000", 2024-09-16T11:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0043978180000000004 2024-09-16T11:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.478456+0000", 2024-09-16T11:43:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1696999999999998e-05 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:43.478475+0000", 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8652999999999999e-05 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:9811 4.2 4:413caf39:::rbd_data.11c14f67cba0.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:47.429227+0000", 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.5092395320000005, 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003851985, 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 9811 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:47.429227+0000", 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:47.429227+0000", 2024-09-16T11:43:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:47.429228+0000", 2024-09-16T11:43:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4400000000000001e-07 2024-09-16T11:43:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:47.429230+0000", 2024-09-16T11:43:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3269999999999999e-06 2024-09-16T11:43:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:47.429236+0000", 2024-09-16T11:43:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8919999999999997e-06 2024-09-16T11:43:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:47.429238+0000", 2024-09-16T11:43:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5160000000000001e-06 2024-09-16T11:43:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:47.429279+0000", 2024-09-16T11:43:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0234000000000001e-05 2024-09-16T11:43:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:47.429324+0000", 2024-09-16T11:43:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5710000000000001e-05 2024-09-16T11:43:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:47.429389+0000", 2024-09-16T11:43:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4214999999999994e-05 2024-09-16T11:43:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:47.429973+0000", 2024-09-16T11:43:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00058464599999999997 2024-09-16T11:43:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:47.433036+0000", 2024-09-16T11:43:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0030625909999999999 2024-09-16T11:43:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:47.433061+0000", 2024-09-16T11:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5327999999999999e-05 2024-09-16T11:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:47.433079+0000", 2024-09-16T11:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8082000000000001e-05 2024-09-16T11:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:12211 4.7 4:e7b2a57b:::rbd_data.11c11284e6d8.0000000000000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:51.481129+0000", 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.45733820400000003, 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048238049999999996, 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 12211 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.481129+0000", 2024-09-16T11:43:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.481129+0000", 2024-09-16T11:43:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.481130+0000", 2024-09-16T11:43:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1599999999999999e-06 2024-09-16T11:43:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.481134+0000", 2024-09-16T11:43:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.262e-06 2024-09-16T11:43:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.481145+0000", 2024-09-16T11:43:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1297999999999999e-05 2024-09-16T11:43:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.481150+0000", 2024-09-16T11:43:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:43:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.481190+0000", 2024-09-16T11:43:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0337999999999999e-05 2024-09-16T11:43:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.481250+0000", 2024-09-16T11:43:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9706000000000001e-05 2024-09-16T11:43:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.481321+0000", 2024-09-16T11:43:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0921999999999996e-05 2024-09-16T11:43:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.481844+0000", 2024-09-16T11:43:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00052338199999999997 2024-09-16T11:43:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.485798+0000", 2024-09-16T11:43:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039536399999999996 2024-09-16T11:43:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.485887+0000", 2024-09-16T11:43:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9751000000000002e-05 2024-09-16T11:43:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.485952+0000", 2024-09-16T11:43:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5022000000000002e-05 2024-09-16T11:43:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:12543 4.2 4:4193ca0c:::rbd_data.11c11284e6d8.000000000000000a:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6566309~41353] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:43:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:51.840360+0000", 2024-09-16T11:43:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.098106378999999994, 2024-09-16T11:43:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0034003319999999998, 2024-09-16T11:43:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 12543 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.840360+0000", 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.840360+0000", 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.840363+0000", 2024-09-16T11:43:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2560000000000001e-06 2024-09-16T11:43:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.840402+0000", 2024-09-16T11:43:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9418000000000001e-05 2024-09-16T11:43:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.840421+0000", 2024-09-16T11:43:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9392e-05 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.840430+0000", 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4659999999999999e-06 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.840491+0000", 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0649000000000002e-05 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.840498+0000", 2024-09-16T11:43:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6380000000000004e-06 2024-09-16T11:43:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.842482+0000", 2024-09-16T11:43:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0019843209999999998 2024-09-16T11:43:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.842567+0000", 2024-09-16T11:43:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4227999999999994e-05 2024-09-16T11:43:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:43:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.842685+0000", 2024-09-16T11:43:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000118033 2024-09-16T11:43:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.843412+0000", 2024-09-16T11:43:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00072768099999999999 2024-09-16T11:43:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.843673+0000", 2024-09-16T11:43:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026014499999999998 2024-09-16T11:43:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.843723+0000", 2024-09-16T11:43:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0306999999999998e-05 2024-09-16T11:43:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.843761+0000", 2024-09-16T11:43:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7798e-05 2024-09-16T11:43:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.063 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:52.065 INFO:teuthology.orchestra.run.smithi070.stdout:2640 write 0x7a60392 thru 0x7a6d830 (0xd49f bytes) 2024-09-16T11:43:52.065 INFO:teuthology.orchestra.run.smithi070.stdout:2641 writesame 0x8a31a05 thru 0x8a32689 data_size 0x1(0xc85 bytes) 2024-09-16T11:43:52.065 INFO:teuthology.orchestra.run.smithi070.stdout:2642 writesame 0x3ef2382 thru 0x3f0167b data_size 0x15(0xf2fa bytes) 2024-09-16T11:43:52.065 INFO:teuthology.orchestra.run.smithi070.stdout:2643 read 0xaaa42d1 thru 0xaab3e06 (0xfb36 bytes) 2024-09-16T11:43:52.065 INFO:teuthology.orchestra.run.smithi070.stdout:2644 read 0x4219f40 thru 0x4223d0d (0x9dce bytes) 2024-09-16T11:43:52.066 INFO:teuthology.orchestra.run.smithi070.stdout:2645 punch from 0x77395a6 to 0x773ec98, (0x56f2 bytes) 2024-09-16T11:43:52.067 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:43:52.072 INFO:teuthology.orchestra.run.smithi070.stdout:2647 write 0xdede4ed thru 0xdeec630 (0xe144 bytes) 2024-09-16T11:43:52.072 INFO:teuthology.orchestra.run.smithi070.stdout:2649 read 0x6571b5 thru 0x65ec5f (0x7aab bytes) 2024-09-16T11:43:52.072 INFO:teuthology.orchestra.run.smithi070.stdout:2650 trunc from 0xdeec631 to 0x9e32d76 2024-09-16T11:43:52.073 INFO:teuthology.orchestra.run.smithi070.stdout:2651 trunc from 0x9e32d76 to 0x141fb5 2024-09-16T11:43:52.073 INFO:teuthology.orchestra.run.smithi070.stdout:2652 write 0x6b18e49 thru 0x6b1f7cd (0x6985 bytes) 2024-09-16T11:43:52.073 INFO:teuthology.orchestra.run.smithi070.stdout:2655 read 0x62285b6 thru 0x62357b2 (0xd1fd bytes) 2024-09-16T11:43:52.073 INFO:teuthology.orchestra.run.smithi070.stdout:2656 punch from 0x37b6227 to 0x37be816, (0x85ef bytes) 2024-09-16T11:43:52.073 INFO:teuthology.orchestra.run.smithi070.stdout:2658 punch from 0xeca113 to 0xed76f0, (0xd5dd bytes) 2024-09-16T11:43:52.073 INFO:teuthology.orchestra.run.smithi070.stdout:2659 read 0x43cd785 thru 0x43d26e2 (0x4f5e bytes) 2024-09-16T11:43:52.074 INFO:teuthology.orchestra.run.smithi070.stdout:2661 write 0x60c9115 thru 0x60cd8d8 (0x47c4 bytes) 2024-09-16T11:43:52.074 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:52.074 INFO:teuthology.orchestra.run.smithi070.stdout:2665 read 0x52e7f81 thru 0x52f62a5 (0xe325 bytes) 2024-09-16T11:43:52.074 INFO:teuthology.orchestra.run.smithi070.stdout:2666 read 0x1b93166 thru 0x1b9d762 (0xa5fd bytes) 2024-09-16T11:43:52.074 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:52.075 INFO:teuthology.orchestra.run.smithi070.stdout:2668 read 0x24a6f02 thru 0x24b261f (0xb71e bytes) 2024-09-16T11:43:52.075 INFO:teuthology.orchestra.run.smithi070.stdout:2669 trunc from 0x6b1f7ce to 0xc2fd619 2024-09-16T11:43:52.075 INFO:teuthology.orchestra.run.smithi070.stdout:2670 writesame 0xd6a1f85 thru 0xd6a3fbd data_size 0x1(0x2039 bytes) 2024-09-16T11:43:52.075 INFO:teuthology.orchestra.run.smithi070.stdout:2672 read 0x973ab26 thru 0x973ac33 (0x10e bytes) 2024-09-16T11:43:52.075 INFO:teuthology.orchestra.run.smithi070.stdout:2674 trunc from 0xd6a3fbe to 0x61962ea 2024-09-16T11:43:52.075 INFO:teuthology.orchestra.run.smithi070.stdout:2676 read 0x5ad5f27 thru 0x5adb64c (0x5726 bytes) 2024-09-16T11:43:52.076 INFO:teuthology.orchestra.run.smithi070.stdout:2677 writesame 0x4bb9a0e thru 0x4bbfb5a data_size 0x39(0x614d bytes) 2024-09-16T11:43:52.076 INFO:teuthology.orchestra.run.smithi070.stdout:2678 writesame 0x853070a thru 0x85376c8 data_size 0x1(0x6fbf bytes) 2024-09-16T11:43:52.076 INFO:teuthology.orchestra.run.smithi070.stdout:2679 read 0x602f2e3 thru 0x603a484 (0xb1a2 bytes) 2024-09-16T11:43:52.076 INFO:teuthology.orchestra.run.smithi070.stdout:2680 read 0x5a24186 thru 0x5a33906 (0xf781 bytes) 2024-09-16T11:43:52.077 INFO:teuthology.orchestra.run.smithi070.stdout:2681 writesame 0xacf1b06 thru 0xacf6ae9 data_size 0x2(0x4fe4 bytes) 2024-09-16T11:43:52.077 INFO:teuthology.orchestra.run.smithi070.stdout:2682 read 0x56c7639 thru 0x56cb2b6 (0x3c7e bytes) 2024-09-16T11:43:52.078 INFO:teuthology.orchestra.run.smithi070.stdout:2683 write 0xc9a484 thru 0xc9f69a (0x5217 bytes) 2024-09-16T11:43:52.080 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:52.073+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:52.082 INFO:teuthology.orchestra.run.smithi070.stdout:2684 trunc from 0xacf6aea to 0x760f6cd 2024-09-16T11:43:52.098 INFO:teuthology.orchestra.run.smithi070.stdout:2687 trunc from 0x760f6cd to 0xd9acf33 2024-09-16T11:43:52.101 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:43:52.101 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:43:52.101 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2024-09-16T11:43:52.101 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .001 2024-09-16T11:43:52.104 INFO:teuthology.orchestra.run.smithi070.stdout:2688 clone 11 order 23 su 4194304 sc 5 2024-09-16T11:43:52.172 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:52.169+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:52.180 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:52.180 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:52.181 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:52.181 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:52.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:43:52.273 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:52.271+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:52.330 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:52.331 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:52.331 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:52.331 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:52.331 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:52.339 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:43:52.373 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:52.371+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:52.475 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:52.472+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:52.478 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:1600 4.1c 4:3a44f897:::rbd_data.11c130c94979.00000000000000ae:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T11:43:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.321435+0000", 2024-09-16T11:43:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.154032523, 2024-09-16T11:43:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001933737, 2024-09-16T11:43:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1600 2024-09-16T11:43:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.321435+0000", 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.321435+0000", 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.321437+0000", 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:43:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.321443+0000", 2024-09-16T11:43:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7750000000000004e-06 2024-09-16T11:43:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.321464+0000", 2024-09-16T11:43:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0191999999999999e-05 2024-09-16T11:43:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.321472+0000", 2024-09-16T11:43:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:43:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.321528+0000", 2024-09-16T11:43:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6378999999999997e-05 2024-09-16T11:43:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.321613+0000", 2024-09-16T11:43:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4967000000000002e-05 2024-09-16T11:43:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.321726+0000", 2024-09-16T11:43:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112791 2024-09-16T11:43:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.322241+0000", 2024-09-16T11:43:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00051583100000000002 2024-09-16T11:43:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.323294+0000", 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001052337 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.323335+0000", 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1362999999999997e-05 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.323368+0000", 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3129000000000001e-05 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4545.0:2270 4.19 e34/28 4:9f0c94cb:::rbd_object_map.11c130c94979.0000000000000006:head v 34'10, mlcod=34'7)", 2024-09-16T11:43:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.891309+0000", 2024-09-16T11:43:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 13.584158589999999, 2024-09-16T11:43:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022415059999999999, 2024-09-16T11:43:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891309+0000", 2024-09-16T11:43:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891309+0000", 2024-09-16T11:43:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891310+0000", 2024-09-16T11:43:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.04e-06 2024-09-16T11:43:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891320+0000", 2024-09-16T11:43:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0773e-05 2024-09-16T11:43:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891339+0000", 2024-09-16T11:43:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8487000000000001e-05 2024-09-16T11:43:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891346+0000", 2024-09-16T11:43:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4410000000000004e-06 2024-09-16T11:43:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891389+0000", 2024-09-16T11:43:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2716000000000002e-05 2024-09-16T11:43:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.891410+0000", 2024-09-16T11:43:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1233000000000002e-05 2024-09-16T11:43:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.893505+0000", 2024-09-16T11:43:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020951699999999999 2024-09-16T11:43:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.893550+0000", 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4645999999999999e-05 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4545.0:2317 4.19 e34/28 4:9f0c94cb:::rbd_object_map.11c130c94979.0000000000000006:head v 34'12, mlcod=34'8)", 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.894697+0000", 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 13.58076984, 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018928339999999999, 2024-09-16T11:43:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.894697+0000", 2024-09-16T11:43:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.894697+0000", 2024-09-16T11:43:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.894698+0000", 2024-09-16T11:43:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:43:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.894702+0000", 2024-09-16T11:43:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4739999999999999e-06 2024-09-16T11:43:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.894711+0000", 2024-09-16T11:43:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8400000000000001e-06 2024-09-16T11:43:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.894716+0000", 2024-09-16T11:43:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7559999999999999e-06 2024-09-16T11:43:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.895051+0000", 2024-09-16T11:43:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00033507300000000002 2024-09-16T11:43:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.895067+0000", 2024-09-16T11:43:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5987000000000001e-05 2024-09-16T11:43:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.896554+0000", 2024-09-16T11:43:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001486809 2024-09-16T11:43:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.896590+0000", 2024-09-16T11:43:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6439000000000001e-05 2024-09-16T11:43:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:2487 4.1c 4:3d6f00f2:::rbd_data.11c130c94979.000000000000001f:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:43:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:38.912100+0000", 2024-09-16T11:43:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 13.563367277999999, 2024-09-16T11:43:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018491009999999999, 2024-09-16T11:43:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 2487 2024-09-16T11:43:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.912100+0000", 2024-09-16T11:43:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.912100+0000", 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.912101+0000", 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.641e-06 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.912107+0000", 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4829999999999999e-06 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.912120+0000", 2024-09-16T11:43:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2587e-05 2024-09-16T11:43:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.912126+0000", 2024-09-16T11:43:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6610000000000003e-06 2024-09-16T11:43:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.912175+0000", 2024-09-16T11:43:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8327e-05 2024-09-16T11:43:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.912259+0000", 2024-09-16T11:43:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4671000000000003e-05 2024-09-16T11:43:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.912444+0000", 2024-09-16T11:43:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000184618 2024-09-16T11:43:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.913454+0000", 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010102170000000001 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.913873+0000", 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00041875199999999999 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.913909+0000", 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6244000000000001e-05 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:38.913949+0000", 2024-09-16T11:43:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9900000000000001e-05 2024-09-16T11:43:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:4697 4.1c 4:3b7ba6fe:::rbd_object_map.11c12816a66b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.536667+0000", 2024-09-16T11:43:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.938799817, 2024-09-16T11:43:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0030708229999999999, 2024-09-16T11:43:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 4697 2024-09-16T11:43:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536667+0000", 2024-09-16T11:43:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536667+0000", 2024-09-16T11:43:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536668+0000", 2024-09-16T11:43:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8599999999999996e-07 2024-09-16T11:43:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536673+0000", 2024-09-16T11:43:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2159999999999997e-06 2024-09-16T11:43:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536687+0000", 2024-09-16T11:43:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3943e-05 2024-09-16T11:43:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536695+0000", 2024-09-16T11:43:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6709999999999999e-06 2024-09-16T11:43:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536730+0000", 2024-09-16T11:43:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4691999999999999e-05 2024-09-16T11:43:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536779+0000", 2024-09-16T11:43:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9109e-05 2024-09-16T11:43:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537019+0000", 2024-09-16T11:43:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00023976999999999999 2024-09-16T11:43:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539622+0000", 2024-09-16T11:43:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0026029880000000001 2024-09-16T11:43:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539659+0000", 2024-09-16T11:43:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6999999999999998e-05 2024-09-16T11:43:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539706+0000", 2024-09-16T11:43:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6875000000000001e-05 2024-09-16T11:43:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539738+0000", 2024-09-16T11:43:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2573000000000001e-05 2024-09-16T11:43:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:4698 4.1c 4:3b7ba6fe:::rbd_object_map.11c12816a66b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.536841+0000", 2024-09-16T11:43:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.938626591, 2024-09-16T11:43:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0030203349999999999, 2024-09-16T11:43:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 4698 2024-09-16T11:43:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536841+0000", 2024-09-16T11:43:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536841+0000", 2024-09-16T11:43:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536841+0000", 2024-09-16T11:43:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0999999999999998e-07 2024-09-16T11:43:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536845+0000", 2024-09-16T11:43:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8369999999999999e-06 2024-09-16T11:43:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536854+0000", 2024-09-16T11:43:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8389999999999996e-06 2024-09-16T11:43:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536861+0000", 2024-09-16T11:43:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6390000000000003e-06 2024-09-16T11:43:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537419+0000", 2024-09-16T11:43:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00055850599999999995 2024-09-16T11:43:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537493+0000", 2024-09-16T11:43:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4375999999999994e-05 2024-09-16T11:43:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537831+0000", 2024-09-16T11:43:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00033727399999999998 2024-09-16T11:43:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539748+0000", 2024-09-16T11:43:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001916882 2024-09-16T11:43:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539784+0000", 2024-09-16T11:43:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5972000000000001e-05 2024-09-16T11:43:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539828+0000", 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4450000000000003e-05 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539861+0000", 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2849999999999999e-05 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:4699 4.1c 4:3b7ba6fe:::rbd_object_map.11c12816a66b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.536944+0000", 2024-09-16T11:43:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.938523234, 2024-09-16T11:43:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003022243, 2024-09-16T11:43:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 4699 2024-09-16T11:43:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536944+0000", 2024-09-16T11:43:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536944+0000", 2024-09-16T11:43:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536945+0000", 2024-09-16T11:43:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2399999999999996e-07 2024-09-16T11:43:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536950+0000", 2024-09-16T11:43:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7960000000000003e-06 2024-09-16T11:43:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536959+0000", 2024-09-16T11:43:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1360000000000003e-06 2024-09-16T11:43:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.536965+0000", 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.538119+0000", 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011536070000000001 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.538205+0000", 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5677e-05 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.538574+0000", 2024-09-16T11:43:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00036919399999999999 2024-09-16T11:43:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539753+0000", 2024-09-16T11:43:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011793470000000001 2024-09-16T11:43:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539896+0000", 2024-09-16T11:43:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014302100000000001 2024-09-16T11:43:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539934+0000", 2024-09-16T11:43:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7487999999999998e-05 2024-09-16T11:43:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539966+0000", 2024-09-16T11:43:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2388000000000002e-05 2024-09-16T11:43:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:4701 4.1c 4:3b7ba6fe:::rbd_object_map.11c12816a66b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.537073+0000", 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.938393779, 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0056192330000000004, 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 4701 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537073+0000", 2024-09-16T11:43:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537073+0000", 2024-09-16T11:43:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537074+0000", 2024-09-16T11:43:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.23e-07 2024-09-16T11:43:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537079+0000", 2024-09-16T11:43:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5050000000000001e-06 2024-09-16T11:43:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537087+0000", 2024-09-16T11:43:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6589999999999996e-06 2024-09-16T11:43:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537092+0000", 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8829999999999997e-06 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.538967+0000", 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018749190000000001 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539055+0000", 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8232000000000005e-05 2024-09-16T11:43:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.539368+0000", 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00031289399999999998 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.542566+0000", 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031977110000000002 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.542611+0000", 2024-09-16T11:43:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4750999999999999e-05 2024-09-16T11:43:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.542658+0000", 2024-09-16T11:43:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7482999999999999e-05 2024-09-16T11:43:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.542693+0000", 2024-09-16T11:43:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4473e-05 2024-09-16T11:43:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:4702 4.1c 4:3b7ba6fe:::rbd_object_map.11c12816a66b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.537113+0000", 2024-09-16T11:43:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.938354023, 2024-09-16T11:43:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0072122280000000002, 2024-09-16T11:43:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 4702 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537113+0000", 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537113+0000", 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537114+0000", 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8999999999999996e-07 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537118+0000", 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5519999999999998e-06 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537125+0000", 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.742e-06 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537129+0000", 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9509999999999999e-06 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.540013+0000", 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002883923 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.540093+0000", 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0117000000000006e-05 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.540411+0000", 2024-09-16T11:43:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000317592 2024-09-16T11:43:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.542739+0000", 2024-09-16T11:43:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0023286600000000002 2024-09-16T11:43:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544267+0000", 2024-09-16T11:43:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001527834 2024-09-16T11:43:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544295+0000", 2024-09-16T11:43:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7555e-05 2024-09-16T11:43:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544325+0000", 2024-09-16T11:43:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0712e-05 2024-09-16T11:43:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:4703 4.1c 4:3b7ba6fe:::rbd_object_map.11c12816a66b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.537223+0000", 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.938244047, 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0071486700000000002, 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 4703 2024-09-16T11:43:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537223+0000", 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537223+0000", 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537224+0000", 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6499999999999999e-07 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537228+0000", 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0960000000000003e-06 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537236+0000", 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.371e-06 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537241+0000", 2024-09-16T11:43:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4920000000000004e-06 2024-09-16T11:43:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.540729+0000", 2024-09-16T11:43:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0034885179999999999 2024-09-16T11:43:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.540808+0000", 2024-09-16T11:43:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8471000000000001e-05 2024-09-16T11:43:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.541175+0000", 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00036780200000000003 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544260+0000", 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0030845569999999999 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544334+0000", 2024-09-16T11:43:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3666999999999998e-05 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544354+0000", 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0440000000000001e-05 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544372+0000", 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7591000000000001e-05 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:4704 4.1c 4:3b7ba6fe:::rbd_object_map.11c12816a66b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.537246+0000", 2024-09-16T11:43:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.938221275, 2024-09-16T11:43:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0073267549999999999, 2024-09-16T11:43:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 4704 2024-09-16T11:43:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537246+0000", 2024-09-16T11:43:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537246+0000", 2024-09-16T11:43:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537246+0000", 2024-09-16T11:43:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5900000000000002e-07 2024-09-16T11:43:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537249+0000", 2024-09-16T11:43:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.43e-06 2024-09-16T11:43:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537263+0000", 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4258e-05 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537268+0000", 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4210000000000003e-06 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.541519+0000", 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0042504509999999997 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.541609+0000", 2024-09-16T11:43:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.967e-05 2024-09-16T11:43:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.542309+0000", 2024-09-16T11:43:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00070017199999999999 2024-09-16T11:43:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544410+0000", 2024-09-16T11:43:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021013809999999998 2024-09-16T11:43:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544479+0000", 2024-09-16T11:43:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8743000000000002e-05 2024-09-16T11:43:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544533+0000", 2024-09-16T11:43:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3780999999999999e-05 2024-09-16T11:43:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544573+0000", 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9988999999999999e-05 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:4705 4.1c 4:3b7ba6fe:::rbd_object_map.11c12816a66b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.537277+0000", 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.938189743000001, 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0073873289999999998, 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 4705 2024-09-16T11:43:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537277+0000", 2024-09-16T11:43:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537277+0000", 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537278+0000", 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7800000000000001e-07 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537282+0000", 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537290+0000", 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4330000000000004e-06 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537294+0000", 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0579999999999999e-06 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.542766+0000", 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0054714969999999996 2024-09-16T11:43:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.542845+0000", 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8814999999999997e-05 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.543150+0000", 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00030515799999999998 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544469+0000", 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001319333 2024-09-16T11:43:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544584+0000", 2024-09-16T11:43:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000115383 2024-09-16T11:43:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544630+0000", 2024-09-16T11:43:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5983999999999999e-05 2024-09-16T11:43:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544665+0000", 2024-09-16T11:43:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4279999999999997e-05 2024-09-16T11:43:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:4706 4.1c 4:3b7ba6fe:::rbd_object_map.11c12816a66b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.537368+0000", 2024-09-16T11:43:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.938099384999999, 2024-09-16T11:43:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0082918139999999998, 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 4706 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537368+0000", 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537368+0000", 2024-09-16T11:43:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537369+0000", 2024-09-16T11:43:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9999999999999996e-07 2024-09-16T11:43:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537374+0000", 2024-09-16T11:43:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9490000000000004e-06 2024-09-16T11:43:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537386+0000", 2024-09-16T11:43:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1989e-05 2024-09-16T11:43:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537391+0000", 2024-09-16T11:43:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8989999999999998e-06 2024-09-16T11:43:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.543520+0000", 2024-09-16T11:43:52.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0061283079999999998 2024-09-16T11:43:52.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.543585+0000", 2024-09-16T11:43:52.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5697000000000005e-05 2024-09-16T11:43:52.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.543930+0000", 2024-09-16T11:43:52.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00034408300000000002 2024-09-16T11:43:52.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.545544+0000", 2024-09-16T11:43:52.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0016142249999999999 2024-09-16T11:43:52.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.545550+0000", 2024-09-16T11:43:52.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7129999999999999e-06 2024-09-16T11:43:52.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.545624+0000", 2024-09-16T11:43:52.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3638000000000002e-05 2024-09-16T11:43:52.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.545660+0000", 2024-09-16T11:43:52.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5413000000000001e-05 2024-09-16T11:43:52.559 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.559 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:4707 4.1c 4:3b7ba6fe:::rbd_object_map.11c12816a66b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.537775+0000", 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.937692349000001, 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0088758069999999994, 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 4707 2024-09-16T11:43:52.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.561 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.561 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537775+0000", 2024-09-16T11:43:52.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.561 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537775+0000", 2024-09-16T11:43:52.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537776+0000", 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6099999999999999e-07 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537782+0000", 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3319999999999997e-06 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537792+0000", 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0324e-05 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.537797+0000", 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1730000000000003e-06 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544712+0000", 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0069142020000000004 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.544796+0000", 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4022000000000003e-05 2024-09-16T11:43:52.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.545169+0000", 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00037329600000000001 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.546324+0000", 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011548870000000001 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.546566+0000", 2024-09-16T11:43:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000242134 2024-09-16T11:43:52.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.546617+0000", 2024-09-16T11:43:52.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1187000000000003e-05 2024-09-16T11:43:52.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.546651+0000", 2024-09-16T11:43:52.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3389000000000002e-05 2024-09-16T11:43:52.565 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.565 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5369 4.1c 4:39cbab95:::rbd_data.11c18a2497a7.0000000000000004:head [stat,truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.025427+0000", 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.450040231999999, 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001771529, 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5369 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.566 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.567 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.025427+0000", 2024-09-16T11:43:52.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.567 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.025427+0000", 2024-09-16T11:43:52.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.025428+0000", 2024-09-16T11:43:52.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4699999999999999e-06 2024-09-16T11:43:52.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.025433+0000", 2024-09-16T11:43:52.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9320000000000004e-06 2024-09-16T11:43:52.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.025450+0000", 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6503000000000001e-05 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.025458+0000", 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3739999999999998e-06 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.025504+0000", 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5633999999999997e-05 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.025583+0000", 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9608000000000001e-05 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.025710+0000", 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012637600000000001 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.026526+0000", 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000816262 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.027118+0000", 2024-09-16T11:43:52.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00059229200000000001 2024-09-16T11:43:52.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.027160+0000", 2024-09-16T11:43:52.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2129999999999998e-05 2024-09-16T11:43:52.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.027198+0000", 2024-09-16T11:43:52.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7948000000000003e-05 2024-09-16T11:43:52.571 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.572 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.572 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.572 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5467 4.1c 4:39cbab95:::rbd_data.11c18a2497a7.0000000000000004:head [stat,truncate 3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.572 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.143896+0000", 2024-09-16T11:43:52.572 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.331571586000001, 2024-09-16T11:43:52.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001753094, 2024-09-16T11:43:52.572 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.572 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.572 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.572 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.572 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.573 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5467 2024-09-16T11:43:52.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.573 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143896+0000", 2024-09-16T11:43:52.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143896+0000", 2024-09-16T11:43:52.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143897+0000", 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.499e-06 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143901+0000", 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3209999999999998e-06 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143922+0000", 2024-09-16T11:43:52.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.107e-05 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143929+0000", 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4570000000000002e-06 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.143985+0000", 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5887999999999998e-05 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144073+0000", 2024-09-16T11:43:52.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7828e-05 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144219+0000", 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014664999999999999 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.144903+0000", 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00068387999999999999 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.145565+0000", 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066171299999999997 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.145619+0000", 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3934000000000002e-05 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.145649+0000", 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9853999999999999e-05 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.577 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.578 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5492 4.1c 4:39cbab95:::rbd_data.11c18a2497a7.0000000000000004:head [stat,truncate 2621440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:43:52.578 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:42.155913+0000", 2024-09-16T11:43:52.578 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 10.319554345, 2024-09-16T11:43:52.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002199463, 2024-09-16T11:43:52.578 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.578 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.578 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.578 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.578 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.578 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5492 2024-09-16T11:43:52.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.579 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155913+0000", 2024-09-16T11:43:52.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155913+0000", 2024-09-16T11:43:52.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155915+0000", 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.491e-06 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155922+0000", 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9589999999999998e-06 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155944+0000", 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2160000000000001e-05 2024-09-16T11:43:52.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.155956+0000", 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1963e-05 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.156008+0000", 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1202000000000002e-05 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.156111+0000", 2024-09-16T11:43:52.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103052 2024-09-16T11:43:52.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.156252+0000", 2024-09-16T11:43:52.582 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014191299999999999 2024-09-16T11:43:52.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.157285+0000", 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001032385 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.158038+0000", 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00075325800000000003 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.158076+0000", 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7397000000000001e-05 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.158112+0000", 2024-09-16T11:43:52.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6683000000000001e-05 2024-09-16T11:43:52.584 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.584 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.584 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.584 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:12515 4.1c 4:3ff441a2:::rbd_data.11c11284e6d8.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-16T11:43:52.584 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:51.781809+0000", 2024-09-16T11:43:52.584 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.69365797500000004, 2024-09-16T11:43:52.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.026874299000000001, 2024-09-16T11:43:52.584 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.584 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.585 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.585 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.585 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.585 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 12515 2024-09-16T11:43:52.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.585 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.585 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.781809+0000", 2024-09-16T11:43:52.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.781809+0000", 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.781811+0000", 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7269999999999999e-06 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.781819+0000", 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1040000000000007e-06 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.587 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.781836+0000", 2024-09-16T11:43:52.587 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6976000000000001e-05 2024-09-16T11:43:52.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.587 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.781842+0000", 2024-09-16T11:43:52.587 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8490000000000002e-06 2024-09-16T11:43:52.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.781886+0000", 2024-09-16T11:43:52.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4168999999999997e-05 2024-09-16T11:43:52.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.781941+0000", 2024-09-16T11:43:52.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5279999999999999e-05 2024-09-16T11:43:52.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.786688+0000", 2024-09-16T11:43:52.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047464619999999999 2024-09-16T11:43:52.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.808566+0000", 2024-09-16T11:43:52.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.021878235999999999 2024-09-16T11:43:52.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.591 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.808580+0000", 2024-09-16T11:43:52.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4318000000000001e-05 2024-09-16T11:43:52.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.591 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.808636+0000", 2024-09-16T11:43:52.592 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5889e-05 2024-09-16T11:43:52.592 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.592 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.592 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.808683+0000", 2024-09-16T11:43:52.592 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7289000000000001e-05 2024-09-16T11:43:52.592 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.592 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.593 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.593 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.4545.0:12515 4.1c e43/28 ondisk, result = 0)", 2024-09-16T11:43:52.593 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:51.800333+0000", 2024-09-16T11:43:52.593 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.67513425599999999, 2024-09-16T11:43:52.594 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0082361120000000003, 2024-09-16T11:43:52.594 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.594 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:52.594 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.594 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.594 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.800333+0000", 2024-09-16T11:43:52.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.800333+0000", 2024-09-16T11:43:52.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.800334+0000", 2024-09-16T11:43:52.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1710000000000001e-06 2024-09-16T11:43:52.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.800339+0000", 2024-09-16T11:43:52.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.673e-06 2024-09-16T11:43:52.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.800354+0000", 2024-09-16T11:43:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.483e-05 2024-09-16T11:43:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.800361+0000", 2024-09-16T11:43:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1949999999999997e-06 2024-09-16T11:43:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.808546+0000", 2024-09-16T11:43:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0081854990000000006 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.808558+0000", 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1391e-05 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.808569+0000", 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1353e-05 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:12555 4.1c 4:3ff441a2:::rbd_data.11c11284e6d8.0000000000000007:head [truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:51.856258+0000", 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.61920924799999999, 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018013090000000001, 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 12555 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:52.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.856258+0000", 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.856258+0000", 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.856260+0000", 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9810000000000002e-06 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.856267+0000", 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0620000000000003e-06 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.856288+0000", 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0990000000000001e-05 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.856298+0000", 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0549e-05 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.856367+0000", 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8678999999999998e-05 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.856447+0000", 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0278000000000006e-05 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.856565+0000", 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000117415 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:43:52.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.857160+0000", 2024-09-16T11:43:52.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00059529899999999998 2024-09-16T11:43:52.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:43:52.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.858011+0000", 2024-09-16T11:43:52.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00085054800000000002 2024-09-16T11:43:52.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:43:52.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.858039+0000", 2024-09-16T11:43:52.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7968999999999999e-05 2024-09-16T11:43:52.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:52.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:52.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:52.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:51.858059+0000", 2024-09-16T11:43:52.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0539000000000001e-05 2024-09-16T11:43:52.604 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.604 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.604 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.604 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:52.604 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:52.604 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:52.605 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:43:52.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:43:52.606 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:52.603+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:52.706 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:52.704+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:52.747 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:52.748 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:52.748 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:52.748 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:52.756 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:43:52.762 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:43:52.762 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:43:52.778 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:43:52.778 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:43:52.794 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:43:52.794 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:43:52.807 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:52.805+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:52.809 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:43:52.809 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:43:52.825 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:43:52.825 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:43:52.838 INFO:tasks.ceph.mon.a.smithi070.stderr:2024-09-16T11:43:52.836+0000 7f4d683f6640 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-16T11:43:52.840 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:43:52.841 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:43:52.846 INFO:teuthology.orchestra.run.smithi070.stderr:osd set-full-ratio 0.001 2024-09-16T11:43:52.861 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:43:52.861 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:43:52.868 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:52.868 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:52.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:43:52.879 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:43:52.879 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:43:52.933 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:52.931+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:52.986 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:52.986 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:52.998 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:43:53.044 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:53.041+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:53.053 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-16T11:43:53.057 INFO:teuthology.orchestra.run.smithi070.stdout:2689 writesame 0x24f77e thru 0x2522da data_size 0x1(0x2b5d bytes) 2024-09-16T11:43:53.068 INFO:teuthology.orchestra.run.smithi070.stdout:2691 punch from 0x914fea6 to 0x915fb8e, (0xfce8 bytes) 2024-09-16T11:43:53.068 INFO:teuthology.orchestra.run.smithi070.stdout:2692 read 0xc8e0808 thru 0xc8ec084 (0xb87d bytes) 2024-09-16T11:43:53.068 INFO:teuthology.orchestra.run.smithi070.stdout:2693 read 0x2b8ad4 thru 0x2bd9d5 (0x4f02 bytes) 2024-09-16T11:43:53.068 INFO:teuthology.orchestra.run.smithi070.stdout:2694 punch from 0xcf27c83 to 0xcf35276, (0xd5f3 bytes) 2024-09-16T11:43:53.068 INFO:teuthology.orchestra.run.smithi070.stdout:2695 write 0x5fb4352 thru 0x5fc1dac (0xda5b bytes) 2024-09-16T11:43:53.077 INFO:teuthology.orchestra.run.smithi070.stdout:2696 write 0xe1cff3f thru 0xe1d2c86 (0x2d48 bytes) 2024-09-16T11:43:53.080 INFO:teuthology.orchestra.run.smithi070.stdout:2697 writesame 0xc09afff thru 0xc0a2f4a data_size 0x2(0x7f4c bytes) 2024-09-16T11:43:53.090 INFO:teuthology.orchestra.run.smithi070.stdout:2699 write 0x6413170 thru 0x641754a (0x43db bytes) 2024-09-16T11:43:53.094 INFO:teuthology.orchestra.run.smithi070.stdout:2700 punch from 0xddb1757 to 0xddb61f9, (0x4aa2 bytes) 2024-09-16T11:43:53.094 INFO:teuthology.orchestra.run.smithi070.stdout:2702 punch from 0x6873c94 to 0x6874e97, (0x1203 bytes) 2024-09-16T11:43:53.094 INFO:teuthology.orchestra.run.smithi070.stdout:2707 writesame 0x41f6694 thru 0x41fb58d data_size 0xb(0x4efa bytes) 2024-09-16T11:43:53.098 INFO:teuthology.orchestra.run.smithi070.stdout:2708 read 0x6034423 thru 0x603dfcb (0x9ba9 bytes) 2024-09-16T11:43:53.099 INFO:teuthology.orchestra.run.smithi070.stdout:2709 read 0xbd331a8 thru 0xbd41aeb (0xe944 bytes) 2024-09-16T11:43:53.099 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:53.099 INFO:teuthology.orchestra.run.smithi070.stdout:2711 writesame 0x8c8adfd thru 0x8c917dd data_size 0x41(0x69e1 bytes) 2024-09-16T11:43:53.104 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:53.104 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:53.109 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:53.107+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:53.113 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:43:53.114 INFO:teuthology.orchestra.run.smithi070.stdout:2712 trunc from 0xe1d2c87 to 0x473a59e 2024-09-16T11:43:53.130 INFO:teuthology.orchestra.run.smithi070.stdout:2715 punch from 0x377349a to 0x377a310, (0x6e76 bytes) 2024-09-16T11:43:53.131 INFO:teuthology.orchestra.run.smithi070.stdout:2718 trunc from 0x473a59e to 0x4474e90 2024-09-16T11:43:53.137 INFO:teuthology.orchestra.run.smithi070.stdout:2719 read 0x2c7af41 thru 0x2c89d83 (0xee43 bytes) 2024-09-16T11:43:53.137 INFO:teuthology.orchestra.run.smithi070.stdout:2720 write 0x1598767 thru 0x15a1317 (0x8bb1 bytes) 2024-09-16T11:43:53.139 INFO:teuthology.orchestra.run.smithi070.stdout:2721 writesame 0x40d9eca thru 0x40dae2b data_size 0xb(0xf62 bytes) 2024-09-16T11:43:53.151 INFO:teuthology.orchestra.run.smithi070.stdout:2722 writesame 0x887bf6b thru 0x888077a data_size 0x8(0x4810 bytes) 2024-09-16T11:43:53.151 INFO:teuthology.orchestra.run.smithi070.stdout:2723 read 0x6bede4e thru 0x6bfdaea (0xfc9d bytes) 2024-09-16T11:43:53.151 INFO:teuthology.orchestra.run.smithi070.stdout:2724 writesame 0x318c277 thru 0x319152a data_size 0x4f(0x52b4 bytes) 2024-09-16T11:43:53.153 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:53.153 INFO:teuthology.orchestra.run.smithi070.stdout:2726 read 0x5d7f859 thru 0x5d82dae (0x3556 bytes) 2024-09-16T11:43:53.153 INFO:teuthology.orchestra.run.smithi070.stdout:2727 punch from 0x3925b15 to 0x3933f86, (0xe471 bytes) 2024-09-16T11:43:53.153 INFO:teuthology.orchestra.run.smithi070.stdout:2728 punch from 0x4b24bdf to 0x4b250e1, (0x502 bytes) 2024-09-16T11:43:53.153 INFO:teuthology.orchestra.run.smithi070.stdout:2729 read 0x4d5a950 thru 0x4d5afa6 (0x657 bytes) 2024-09-16T11:43:53.166 INFO:teuthology.orchestra.run.smithi070.stdout:2730 writesame 0xed18ff9 thru 0xed2184b data_size 0x1(0x8853 bytes) 2024-09-16T11:43:53.166 INFO:teuthology.orchestra.run.smithi070.stdout:2731 punch from 0x5cdc24f to 0x5ce82f7, (0xc0a8 bytes) 2024-09-16T11:43:53.166 INFO:teuthology.orchestra.run.smithi070.stdout:2734 write 0x3867cd0 thru 0x386ed81 (0x70b2 bytes) 2024-09-16T11:43:53.167 INFO:teuthology.orchestra.run.smithi070.stdout:2735 read 0xa577c0f thru 0xa580734 (0x8b26 bytes) 2024-09-16T11:43:53.167 INFO:teuthology.orchestra.run.smithi070.stdout:2737 write 0x1d205f0 thru 0x1d23881 (0x3292 bytes) 2024-09-16T11:43:53.170 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:53.170 INFO:teuthology.orchestra.run.smithi070.stdout:2740 write 0xe17e3e thru 0xe20030 (0x81f3 bytes) 2024-09-16T11:43:53.174 INFO:teuthology.orchestra.run.smithi070.stdout:2741 clone 12 order 20 su 131072 sc 3 2024-09-16T11:43:53.210 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:53.208+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:53.252 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:53.252 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:53.260 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:43:53.311 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:53.310+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:53.399 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:53.399 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:53.407 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:43:53.412 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:53.411+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:53.505 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:53.505 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:53.512 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:53.510+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:53.514 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:43:53.613 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:53.611+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:53.653 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:53.653 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:53.662 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:43:53.714 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:53.713+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:53.799 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:53.800 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:53.808 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:43:53.851 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:53.849+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:53.860 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:43:53.861 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2024-09-16T11:43:53.861 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .95 2024-09-16T11:43:53.907 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone11 from 0xed2184c (overlap 0x4474e90) to 0x0 2024-09-16T11:43:53.911 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:53.911 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:53.916 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:53.914+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:53.920 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:43:53.952 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-16T11:43:54.017 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:54.015+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:54.058 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:54.059 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:54.067 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:43:54.118 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:54.116+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:54.165 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:54.165 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:54.173 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:43:54.195 INFO:teuthology.orchestra.run.smithi070.stdout:2742 trunc from 0xed2184c to 0xbdf563a 2024-09-16T11:43:54.208 INFO:teuthology.orchestra.run.smithi070.stdout:2744 read 0x3a04a66 thru 0x3a0c66a (0x7c05 bytes) 2024-09-16T11:43:54.208 INFO:teuthology.orchestra.run.smithi070.stdout:2745 write 0xb6e32d9 thru 0xb6ecf57 (0x9c7f bytes) 2024-09-16T11:43:54.211 INFO:teuthology.orchestra.run.smithi070.stdout:2746 writesame 0x4805d25 thru 0x480809b data_size 0x1(0x2377 bytes) 2024-09-16T11:43:54.218 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:54.218 INFO:teuthology.orchestra.run.smithi070.stdout:2749 read 0xab93e72 thru 0xab95ee7 (0x2076 bytes) 2024-09-16T11:43:54.218 INFO:teuthology.orchestra.run.smithi070.stdout:2750 read 0x7b3ed thru 0x87dc3 (0xc9d7 bytes) 2024-09-16T11:43:54.219 INFO:teuthology.orchestra.run.smithi070.stdout:2752 read 0x7002a7c thru 0x700b6c5 (0x8c4a bytes) 2024-09-16T11:43:54.219 INFO:teuthology.orchestra.run.smithi070.stdout:2753 punch from 0x88585c5 to 0x8864c6f, (0xc6aa bytes) 2024-09-16T11:43:54.219 INFO:teuthology.orchestra.run.smithi070.stdout:2754 writesame 0x2f56d0e thru 0x2f5f020 data_size 0xf(0x8313 bytes) 2024-09-16T11:43:54.224 INFO:teuthology.orchestra.run.smithi070.stdout:2758 read 0x9b48825 thru 0x9b4a8c5 (0x20a1 bytes) 2024-09-16T11:43:54.224 INFO:teuthology.orchestra.run.smithi070.stdout:2762 write 0x849bae1 thru 0x84a0c7d (0x519d bytes) 2024-09-16T11:43:54.227 INFO:teuthology.orchestra.run.smithi070.stdout:2763 punch from 0x548a9f6 to 0x5496d87, (0xc391 bytes) 2024-09-16T11:43:54.228 INFO:teuthology.orchestra.run.smithi070.stdout:2764 read 0x1a4cf72 thru 0x1a58618 (0xb6a7 bytes) 2024-09-16T11:43:54.234 INFO:teuthology.orchestra.run.smithi070.stdout:2765 write 0xd0fad13 thru 0xd0fd882 (0x2b70 bytes) 2024-09-16T11:43:54.236 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:54.242 INFO:teuthology.orchestra.run.smithi070.stdout:2767 trunc from 0xd0fd883 to 0x7f5c07d 2024-09-16T11:43:54.258 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:54.255+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:54.267 INFO:teuthology.orchestra.run.smithi070.stdout:2768 write 0xb5337b8 thru 0xb540e8b (0xd6d4 bytes) 2024-09-16T11:43:54.282 INFO:teuthology.orchestra.run.smithi070.stdout:2769 trunc from 0xb540e8c to 0x4ceae2c 2024-09-16T11:43:54.297 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:54.314 INFO:teuthology.orchestra.run.smithi070.stdout:2771 write 0xeb5418d thru 0xeb5edb5 (0xac29 bytes) 2024-09-16T11:43:54.316 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:54.316 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:54.316 INFO:teuthology.orchestra.run.smithi070.stdout:2772 read 0x891937 thru 0x89b6c5 (0x9d8f bytes) 2024-09-16T11:43:54.317 INFO:teuthology.orchestra.run.smithi070.stdout:2773 writesame 0xe5de6d2 thru 0xe5e828b data_size 0x1f(0x9bba bytes) 2024-09-16T11:43:54.319 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:54.317+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:54.320 INFO:teuthology.orchestra.run.smithi070.stdout:2774 write 0x23fb20 thru 0x242cb2 (0x3193 bytes) 2024-09-16T11:43:54.325 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:43:54.334 INFO:teuthology.orchestra.run.smithi070.stdout:2775 trunc from 0xeb5edb6 to 0x6f48096 2024-09-16T11:43:54.352 INFO:teuthology.orchestra.run.smithi070.stdout:2776 writesame 0x7d003cb thru 0x7d0d294 data_size 0x33(0xceca bytes) 2024-09-16T11:43:54.353 INFO:teuthology.orchestra.run.smithi070.stdout:2777 read 0x6659acf thru 0x6660459 (0x698b bytes) 2024-09-16T11:43:54.353 INFO:teuthology.orchestra.run.smithi070.stdout:2778 punch from 0x4e4144e to 0x4e45937, (0x44e9 bytes) 2024-09-16T11:43:54.353 INFO:teuthology.orchestra.run.smithi070.stdout:2779 clone 13 order 21 su 8192 sc 13 2024-09-16T11:43:54.419 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:54.417+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:54.464 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:54.465 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:54.473 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:43:54.520 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:54.519+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:54.612 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:54.612 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:54.620 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:43:54.621 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:54.619+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:54.723 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:54.721+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:54.728 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:43:54.824 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:54.822+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:54.856 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T11:43:54.856 INFO:teuthology.orchestra.run.smithi070.stderr:osd set-full-ratio 0.95 2024-09-16T11:43:54.876 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:54.877 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:54.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:43:54.892 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:43:54.918 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone12 from 0x7d0d295 (overlap 0x4ceae2c) to 0x3d88b56 2024-09-16T11:43:54.961 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:54.959+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:55.018 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-16T11:43:55.019 INFO:teuthology.orchestra.run.smithi070.stdout:2780 punch from 0x28394aa to 0x283a623, (0x1179 bytes) 2024-09-16T11:43:55.022 INFO:teuthology.orchestra.run.smithi070.stdout:2781 read 0x3ea720b thru 0x3eb013f (0x8f35 bytes) 2024-09-16T11:43:55.022 INFO:teuthology.orchestra.run.smithi070.stdout:2782 punch from 0x362a92d to 0x362cd3d, (0x2410 bytes) 2024-09-16T11:43:55.022 INFO:teuthology.orchestra.run.smithi070.stdout:2783 writesame 0x16dea17 thru 0x16e33b9 data_size 0x1(0x49a3 bytes) 2024-09-16T11:43:55.026 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:55.024+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:55.035 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:55.035 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:55.049 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:43:55.061 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:43:55.062 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:43:55.068 INFO:teuthology.orchestra.run.smithi070.stdout:2784 read 0x3fe33c3 thru 0x3fe5b1b (0x2759 bytes) 2024-09-16T11:43:55.070 INFO:teuthology.orchestra.run.smithi070.stdout:2786 trunc from 0x7d0d295 to 0x98d702c 2024-09-16T11:43:55.080 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:43:55.080 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:43:55.082 INFO:teuthology.orchestra.run.smithi070.stdout:2788 trunc from 0x98d702c to 0xe9e863f 2024-09-16T11:43:55.087 INFO:teuthology.orchestra.run.smithi070.stdout:2790 read 0xe0410c8 thru 0xe041911 (0x84a bytes) 2024-09-16T11:43:55.087 INFO:teuthology.orchestra.run.smithi070.stdout:2791 read 0x46acfce thru 0x46b348b (0x64be bytes) 2024-09-16T11:43:55.088 INFO:teuthology.orchestra.run.smithi070.stdout:2793 write 0x843ceb4 thru 0x8440c87 (0x3dd4 bytes) 2024-09-16T11:43:55.092 INFO:teuthology.orchestra.run.smithi070.stdout:2794 read 0x12cf7f3 thru 0x12db41e (0xbc2c bytes) 2024-09-16T11:43:55.094 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:55.094 INFO:teuthology.orchestra.run.smithi070.stdout:2797 write 0x4dd5916 thru 0x4de34c6 (0xdbb1 bytes) 2024-09-16T11:43:55.112 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:43:55.112 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:43:55.128 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:55.125+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:55.140 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:43:55.140 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:43:55.160 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:43:55.160 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:43:55.182 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:43:55.182 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:43:55.207 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:55.207 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:55.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.207 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15794 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.207 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.192117+0000", 2024-09-16T11:43:55.207 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.004855336, 2024-09-16T11:43:55.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048796070000000002, 2024-09-16T11:43:55.207 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.208 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.208 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.208 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.208 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.208 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15794 2024-09-16T11:43:55.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.208 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192117+0000", 2024-09-16T11:43:55.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192117+0000", 2024-09-16T11:43:55.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192118+0000", 2024-09-16T11:43:55.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3500000000000002e-07 2024-09-16T11:43:55.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192123+0000", 2024-09-16T11:43:55.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3190000000000002e-06 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192126+0000", 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192128+0000", 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9379999999999999e-06 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15827 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.192744+0000", 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0042283279999999999, 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0042886139999999996, 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15827 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192744+0000", 2024-09-16T11:43:55.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192744+0000", 2024-09-16T11:43:55.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192744+0000", 2024-09-16T11:43:55.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.17e-07 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192745+0000", 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0279999999999999e-06 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192750+0000", 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3499999999999999e-06 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.213 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192752+0000", 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4250000000000001e-06 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15861 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.193319+0000", 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.003653487, 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037418130000000001, 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.214 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15861 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193319+0000", 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193319+0000", 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193319+0000", 2024-09-16T11:43:55.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.53e-07 2024-09-16T11:43:55.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193320+0000", 2024-09-16T11:43:55.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.850000000000001e-07 2024-09-16T11:43:55.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193323+0000", 2024-09-16T11:43:55.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7250000000000002e-06 2024-09-16T11:43:55.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193325+0000", 2024-09-16T11:43:55.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7489999999999999e-06 2024-09-16T11:43:55.217 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.217 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.217 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.217 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15943 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.217 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.193846+0000", 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0031264489999999999, 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0032429070000000002, 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15943 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193846+0000", 2024-09-16T11:43:55.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193846+0000", 2024-09-16T11:43:55.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193846+0000", 2024-09-16T11:43:55.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9499999999999998e-07 2024-09-16T11:43:55.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193851+0000", 2024-09-16T11:43:55.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:43:55.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193854+0000", 2024-09-16T11:43:55.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-06 2024-09-16T11:43:55.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193856+0000", 2024-09-16T11:43:55.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:43:55.221 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.221 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.221 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.221 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16004 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.221 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.194284+0000", 2024-09-16T11:43:55.221 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0026886359999999999, 2024-09-16T11:43:55.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0028458609999999999, 2024-09-16T11:43:55.221 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.221 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.221 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.221 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.222 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.222 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16004 2024-09-16T11:43:55.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.222 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194284+0000", 2024-09-16T11:43:55.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194284+0000", 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194284+0000", 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1400000000000001e-07 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194285+0000", 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6499999999999998e-07 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194288+0000", 2024-09-16T11:43:55.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8439999999999998e-06 2024-09-16T11:43:55.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194294+0000", 2024-09-16T11:43:55.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0569999999999998e-06 2024-09-16T11:43:55.224 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.224 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.224 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.224 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15588 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.225 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195564+0000", 2024-09-16T11:43:55.225 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0014082420000000001, 2024-09-16T11:43:55.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015940489999999999, 2024-09-16T11:43:55.225 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.225 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.225 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.225 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.225 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.225 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15588 2024-09-16T11:43:55.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.225 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195564+0000", 2024-09-16T11:43:55.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195564+0000", 2024-09-16T11:43:55.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195564+0000", 2024-09-16T11:43:55.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.23e-07 2024-09-16T11:43:55.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195566+0000", 2024-09-16T11:43:55.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.42e-06 2024-09-16T11:43:55.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195570+0000", 2024-09-16T11:43:55.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2950000000000004e-06 2024-09-16T11:43:55.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195573+0000", 2024-09-16T11:43:55.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9519999999999999e-06 2024-09-16T11:43:55.228 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.228 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.228 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.228 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15630 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.228 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196304+0000", 2024-09-16T11:43:55.228 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00066800200000000001, 2024-09-16T11:43:55.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00088182799999999997, 2024-09-16T11:43:55.228 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.229 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.229 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.229 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.229 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.229 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15630 2024-09-16T11:43:55.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.229 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196304+0000", 2024-09-16T11:43:55.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196304+0000", 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196305+0000", 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196323+0000", 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8578999999999999e-05 2024-09-16T11:43:55.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196330+0000", 2024-09-16T11:43:55.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2720000000000003e-06 2024-09-16T11:43:55.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196333+0000", 2024-09-16T11:43:55.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.715e-06 2024-09-16T11:43:55.231 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.231 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.231 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15664 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196961+0000", 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.1581e-05, 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000253033, 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15664 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196961+0000", 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196961+0000", 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196961+0000", 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7e-07 2024-09-16T11:43:55.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196963+0000", 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.418e-06 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196968+0000", 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6010000000000002e-06 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196971+0000", 2024-09-16T11:43:55.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0089999999999999e-06 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15828 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.192754+0000", 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0042183949999999998, 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0044880069999999996, 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.235 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15828 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192754+0000", 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192754+0000", 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192754+0000", 2024-09-16T11:43:55.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0599999999999999e-07 2024-09-16T11:43:55.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192755+0000", 2024-09-16T11:43:55.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.850000000000001e-07 2024-09-16T11:43:55.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192758+0000", 2024-09-16T11:43:55.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.666e-06 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192760+0000", 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7749999999999999e-06 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15862 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.193327+0000", 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0036458660000000002, 2024-09-16T11:43:55.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039446949999999998, 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15862 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193327+0000", 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193327+0000", 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193327+0000", 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0100000000000001e-07 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193328+0000", 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6600000000000005e-07 2024-09-16T11:43:55.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193330+0000", 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7149999999999998e-06 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193336+0000", 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3029999999999999e-06 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16005 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.194296+0000", 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0026767850000000001, 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003003159, 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16005 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194296+0000", 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194296+0000", 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194296+0000", 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2000000000000001e-07 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194297+0000", 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8899999999999998e-07 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194300+0000", 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.92e-06 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194302+0000", 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:43:55.244 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15589 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195585+0000", 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001387711, 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017420700000000001, 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.245 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15589 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195585+0000", 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195585+0000", 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195585+0000", 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9999999999999998e-07 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195587+0000", 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.61e-06 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195593+0000", 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0310000000000004e-06 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195596+0000", 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9569999999999998e-06 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15631 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196337+0000", 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00063501300000000003, 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010158330000000001, 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.248 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15631 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196337+0000", 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196337+0000", 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196338+0000", 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5400000000000001e-07 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196340+0000", 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.733e-06 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196346+0000", 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7660000000000004e-06 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196348+0000", 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9560000000000002e-06 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15665 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.251 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196987+0000", 2024-09-16T11:43:55.252 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9999852, 2024-09-16T11:43:55.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00039376199999999997, 2024-09-16T11:43:55.252 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.252 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.252 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.252 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.252 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.252 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15665 2024-09-16T11:43:55.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.252 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196987+0000", 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196987+0000", 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196988+0000", 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3099999999999999e-07 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196989+0000", 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.234e-06 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196993+0000", 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196996+0000", 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4189999999999998e-06 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.254 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15829 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.192773+0000", 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0041998890000000001, 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046362499999999997, 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15829 2024-09-16T11:43:55.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.256 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192773+0000", 2024-09-16T11:43:55.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192773+0000", 2024-09-16T11:43:55.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192773+0000", 2024-09-16T11:43:55.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:43:55.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192774+0000", 2024-09-16T11:43:55.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2389999999999999e-06 2024-09-16T11:43:55.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192777+0000", 2024-09-16T11:43:55.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9490000000000001e-06 2024-09-16T11:43:55.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192779+0000", 2024-09-16T11:43:55.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7579999999999999e-06 2024-09-16T11:43:55.258 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.258 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.258 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.258 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15863 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.258 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.193338+0000", 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0036347440000000001, 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0040974840000000002, 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15863 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193338+0000", 2024-09-16T11:43:55.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193338+0000", 2024-09-16T11:43:55.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193338+0000", 2024-09-16T11:43:55.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:43:55.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193339+0000", 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2800000000000005e-07 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193342+0000", 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8499999999999998e-06 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193344+0000", 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.765e-06 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.261 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16006 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.194304+0000", 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0026688660000000002, 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031580829999999999, 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16006 2024-09-16T11:43:55.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.263 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194304+0000", 2024-09-16T11:43:55.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194304+0000", 2024-09-16T11:43:55.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194304+0000", 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0200000000000001e-07 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194305+0000", 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1900000000000001e-07 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194308+0000", 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.242e-06 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194310+0000", 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8309999999999999e-06 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15590 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195612+0000", 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001360514, 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001876588, 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.265 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.266 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.266 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.266 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.266 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15590 2024-09-16T11:43:55.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.266 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195612+0000", 2024-09-16T11:43:55.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195612+0000", 2024-09-16T11:43:55.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195612+0000", 2024-09-16T11:43:55.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4900000000000002e-07 2024-09-16T11:43:55.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195614+0000", 2024-09-16T11:43:55.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.421e-06 2024-09-16T11:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195617+0000", 2024-09-16T11:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195621+0000", 2024-09-16T11:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.268 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15632 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196363+0000", 2024-09-16T11:43:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00060968600000000002, 2024-09-16T11:43:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011567210000000001, 2024-09-16T11:43:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15632 2024-09-16T11:43:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196363+0000", 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196363+0000", 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196363+0000", 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196365+0000", 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4270000000000001e-06 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196370+0000", 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8520000000000003e-06 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196373+0000", 2024-09-16T11:43:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:43:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15666 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197010+0000", 2024-09-16T11:43:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9999628, 2024-09-16T11:43:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00053702899999999996, 2024-09-16T11:43:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15666 2024-09-16T11:43:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197010+0000", 2024-09-16T11:43:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197010+0000", 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197010+0000", 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197012+0000", 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5969999999999999e-06 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197016+0000", 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.391e-06 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197020+0000", 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7400000000000002e-06 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15830 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.192786+0000", 2024-09-16T11:43:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0041861850000000003, 2024-09-16T11:43:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047879649999999999, 2024-09-16T11:43:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15830 2024-09-16T11:43:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192786+0000", 2024-09-16T11:43:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192786+0000", 2024-09-16T11:43:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192787+0000", 2024-09-16T11:43:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.34e-07 2024-09-16T11:43:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192788+0000", 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2389999999999999e-06 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192791+0000", 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9490000000000001e-06 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.192793+0000", 2024-09-16T11:43:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:43:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15865 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.193345+0000", 2024-09-16T11:43:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0036270439999999998, 2024-09-16T11:43:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0042553000000000001, 2024-09-16T11:43:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15865 2024-09-16T11:43:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193345+0000", 2024-09-16T11:43:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193345+0000", 2024-09-16T11:43:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193346+0000", 2024-09-16T11:43:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0800000000000001e-07 2024-09-16T11:43:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193347+0000", 2024-09-16T11:43:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4600000000000003e-07 2024-09-16T11:43:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193349+0000", 2024-09-16T11:43:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.689e-06 2024-09-16T11:43:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193351+0000", 2024-09-16T11:43:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.815e-06 2024-09-16T11:43:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16007 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.194312+0000", 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.002660918, 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0033160300000000002, 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16007 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194312+0000", 2024-09-16T11:43:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194312+0000", 2024-09-16T11:43:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194312+0000", 2024-09-16T11:43:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.35e-07 2024-09-16T11:43:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194313+0000", 2024-09-16T11:43:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4399999999999999e-07 2024-09-16T11:43:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194317+0000", 2024-09-16T11:43:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5900000000000001e-06 2024-09-16T11:43:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194320+0000", 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9349999999999999e-06 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15591 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195625+0000", 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0013470299999999999, 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020300990000000001, 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15591 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195625+0000", 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195625+0000", 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195626+0000", 2024-09-16T11:43:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8599999999999998e-07 2024-09-16T11:43:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195628+0000", 2024-09-16T11:43:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:43:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195633+0000", 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3700000000000003e-06 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195636+0000", 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2260000000000001e-06 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15634 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196377+0000", 2024-09-16T11:43:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00059577199999999995, 2024-09-16T11:43:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001306064, 2024-09-16T11:43:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15634 2024-09-16T11:43:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196377+0000", 2024-09-16T11:43:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196377+0000", 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196377+0000", 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1199999999999999e-07 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196379+0000", 2024-09-16T11:43:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4869999999999999e-06 2024-09-16T11:43:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196384+0000", 2024-09-16T11:43:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:43:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196392+0000", 2024-09-16T11:43:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7500000000000003e-06 2024-09-16T11:43:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15667 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197024+0000", 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9999485, 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00068615500000000003, 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15667 2024-09-16T11:43:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197024+0000", 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197024+0000", 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197024+0000", 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.13e-07 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197026+0000", 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3740000000000001e-06 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197030+0000", 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3429999999999998e-06 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197038+0000", 2024-09-16T11:43:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2730000000000002e-06 2024-09-16T11:43:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15866 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.193365+0000", 2024-09-16T11:43:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0036074010000000001, 2024-09-16T11:43:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0043725760000000004, 2024-09-16T11:43:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15866 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193365+0000", 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193365+0000", 2024-09-16T11:43:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193365+0000", 2024-09-16T11:43:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.03e-07 2024-09-16T11:43:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193367+0000", 2024-09-16T11:43:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2589999999999999e-06 2024-09-16T11:43:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193370+0000", 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9890000000000001e-06 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193372+0000", 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.872e-06 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16008 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.194322+0000", 2024-09-16T11:43:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0026503270000000001, 2024-09-16T11:43:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0034423740000000002, 2024-09-16T11:43:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16008 2024-09-16T11:43:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194322+0000", 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194322+0000", 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194322+0000", 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194327+0000", 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4510000000000002e-06 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194332+0000", 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194336+0000", 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7900000000000001e-06 2024-09-16T11:43:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15593 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195642+0000", 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0013306189999999999, 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002149209, 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15593 2024-09-16T11:43:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195642+0000", 2024-09-16T11:43:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195642+0000", 2024-09-16T11:43:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195642+0000", 2024-09-16T11:43:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1299999999999999e-07 2024-09-16T11:43:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195643+0000", 2024-09-16T11:43:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.062e-06 2024-09-16T11:43:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195646+0000", 2024-09-16T11:43:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1999999999999999e-06 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195655+0000", 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2640000000000008e-06 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15635 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196396+0000", 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00057695200000000004, 2024-09-16T11:43:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0014219860000000001, 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15635 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196396+0000", 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196396+0000", 2024-09-16T11:43:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196396+0000", 2024-09-16T11:43:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:43:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196397+0000", 2024-09-16T11:43:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6360000000000001e-06 2024-09-16T11:43:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196403+0000", 2024-09-16T11:43:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3340000000000001e-06 2024-09-16T11:43:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196406+0000", 2024-09-16T11:43:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.394e-06 2024-09-16T11:43:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15668 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197042+0000", 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9999304, 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00080242400000000004, 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15668 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197042+0000", 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197042+0000", 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197043+0000", 2024-09-16T11:43:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:43:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197048+0000", 2024-09-16T11:43:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4190000000000001e-06 2024-09-16T11:43:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197053+0000", 2024-09-16T11:43:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3240000000000002e-06 2024-09-16T11:43:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197055+0000", 2024-09-16T11:43:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:43:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15868 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.193385+0000", 2024-09-16T11:43:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0035879789999999998, 2024-09-16T11:43:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004487207, 2024-09-16T11:43:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15868 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193385+0000", 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193385+0000", 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193385+0000", 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193386+0000", 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.234e-06 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193389+0000", 2024-09-16T11:43:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9040000000000001e-06 2024-09-16T11:43:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193391+0000", 2024-09-16T11:43:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.748e-06 2024-09-16T11:43:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16009 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.194340+0000", 2024-09-16T11:43:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0026328850000000002, 2024-09-16T11:43:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003558931, 2024-09-16T11:43:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16009 2024-09-16T11:43:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194340+0000", 2024-09-16T11:43:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194340+0000", 2024-09-16T11:43:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194340+0000", 2024-09-16T11:43:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.39e-07 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194342+0000", 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.598e-06 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194347+0000", 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0590000000000002e-06 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194351+0000", 2024-09-16T11:43:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.126e-06 2024-09-16T11:43:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15594 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195670+0000", 2024-09-16T11:43:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0013023329999999999, 2024-09-16T11:43:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022546939999999998, 2024-09-16T11:43:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15594 2024-09-16T11:43:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195670+0000", 2024-09-16T11:43:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195670+0000", 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195671+0000", 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.13e-07 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195673+0000", 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8670000000000001e-06 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195679+0000", 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1569999999999997e-06 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195682+0000", 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1089999999999998e-06 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15636 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196409+0000", 2024-09-16T11:43:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00056363400000000003, 2024-09-16T11:43:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015423730000000001, 2024-09-16T11:43:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15636 2024-09-16T11:43:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196409+0000", 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196409+0000", 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196409+0000", 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196411+0000", 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.575e-06 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196416+0000", 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3249999999999998e-06 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196419+0000", 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0460000000000001e-06 2024-09-16T11:43:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15669 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197059+0000", 2024-09-16T11:43:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9999132, 2024-09-16T11:43:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00092499900000000003, 2024-09-16T11:43:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15669 2024-09-16T11:43:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197059+0000", 2024-09-16T11:43:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197059+0000", 2024-09-16T11:43:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197060+0000", 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197061+0000", 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4219999999999999e-06 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197064+0000", 2024-09-16T11:43:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0589999999999998e-06 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197066+0000", 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9439999999999999e-06 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15870 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.193404+0000", 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0035688209999999998, 2024-09-16T11:43:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046088809999999996, 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15870 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193404+0000", 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193404+0000", 2024-09-16T11:43:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193404+0000", 2024-09-16T11:43:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:43:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193405+0000", 2024-09-16T11:43:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.226e-06 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193408+0000", 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.836e-06 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193410+0000", 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.821e-06 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16010 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.194354+0000", 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0026181949999999998, 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0036874329999999999, 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16010 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194354+0000", 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194354+0000", 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194355+0000", 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:43:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194356+0000", 2024-09-16T11:43:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5650000000000001e-06 2024-09-16T11:43:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194362+0000", 2024-09-16T11:43:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3199999999999999e-06 2024-09-16T11:43:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194365+0000", 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8480000000000003e-06 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15595 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195698+0000", 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0012749619999999999, 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0023708319999999998, 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15595 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195698+0000", 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195698+0000", 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195698+0000", 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2999999999999999e-07 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195699+0000", 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3960000000000001e-06 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195702+0000", 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2040000000000002e-06 2024-09-16T11:43:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195706+0000", 2024-09-16T11:43:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1580000000000001e-06 2024-09-16T11:43:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15637 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196423+0000", 2024-09-16T11:43:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00054994099999999997, 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0016799619999999999, 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15637 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196423+0000", 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196423+0000", 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196423+0000", 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196424+0000", 2024-09-16T11:43:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4610000000000001e-06 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196430+0000", 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8540000000000002e-06 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196433+0000", 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0599999999999999e-06 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15670 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197074+0000", 2024-09-16T11:43:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9998989, 2024-09-16T11:43:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010569049999999999, 2024-09-16T11:43:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15670 2024-09-16T11:43:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197074+0000", 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197074+0000", 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197074+0000", 2024-09-16T11:43:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0200000000000003e-07 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197075+0000", 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2139999999999999e-06 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197078+0000", 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9660000000000001e-06 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197080+0000", 2024-09-16T11:43:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.874e-06 2024-09-16T11:43:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15871 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.193423+0000", 2024-09-16T11:43:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0035493870000000002, 2024-09-16T11:43:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047366070000000003, 2024-09-16T11:43:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15871 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193423+0000", 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193423+0000", 2024-09-16T11:43:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193423+0000", 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9400000000000001e-07 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193425+0000", 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2619999999999999e-06 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193428+0000", 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9380000000000001e-06 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.193430+0000", 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.984e-06 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16011 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.194369+0000", 2024-09-16T11:43:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.002603785, 2024-09-16T11:43:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0038199689999999999, 2024-09-16T11:43:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16011 2024-09-16T11:43:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194369+0000", 2024-09-16T11:43:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194369+0000", 2024-09-16T11:43:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194369+0000", 2024-09-16T11:43:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2800000000000003e-07 2024-09-16T11:43:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194370+0000", 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.471e-06 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194376+0000", 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2580000000000003e-06 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194380+0000", 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9389999999999999e-06 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15596 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195709+0000", 2024-09-16T11:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001263742, 2024-09-16T11:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002507455, 2024-09-16T11:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15596 2024-09-16T11:43:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195709+0000", 2024-09-16T11:43:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195709+0000", 2024-09-16T11:43:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195709+0000", 2024-09-16T11:43:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7100000000000002e-07 2024-09-16T11:43:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195711+0000", 2024-09-16T11:43:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4139999999999999e-06 2024-09-16T11:43:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195716+0000", 2024-09-16T11:43:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2900000000000002e-06 2024-09-16T11:43:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195719+0000", 2024-09-16T11:43:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1499999999999999e-06 2024-09-16T11:43:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15638 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196450+0000", 2024-09-16T11:43:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00052263500000000003, 2024-09-16T11:43:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001794148, 2024-09-16T11:43:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15638 2024-09-16T11:43:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196450+0000", 2024-09-16T11:43:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196450+0000", 2024-09-16T11:43:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196450+0000", 2024-09-16T11:43:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:43:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196452+0000", 2024-09-16T11:43:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:43:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196457+0000", 2024-09-16T11:43:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2290000000000002e-06 2024-09-16T11:43:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196460+0000", 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1020000000000001e-06 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15671 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197082+0000", 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9998903, 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011894399999999999, 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15671 2024-09-16T11:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197082+0000", 2024-09-16T11:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197082+0000", 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197083+0000", 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197083+0000", 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.71e-07 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197086+0000", 2024-09-16T11:43:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.79e-06 2024-09-16T11:43:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197088+0000", 2024-09-16T11:43:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0990000000000001e-06 2024-09-16T11:43:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16012 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.194397+0000", 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0025751469999999999, 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003904336, 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16012 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.365 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194397+0000", 2024-09-16T11:43:55.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194397+0000", 2024-09-16T11:43:55.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194398+0000", 2024-09-16T11:43:55.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3099999999999998e-07 2024-09-16T11:43:55.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194402+0000", 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1659999999999996e-06 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194408+0000", 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0399999999999998e-06 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194412+0000", 2024-09-16T11:43:55.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.546e-06 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15597 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195722+0000", 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001250026, 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0026290200000000001, 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.368 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.369 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15597 2024-09-16T11:43:55.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.369 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195722+0000", 2024-09-16T11:43:55.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195722+0000", 2024-09-16T11:43:55.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.370 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.370 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195723+0000", 2024-09-16T11:43:55.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:43:55.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.370 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.370 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195732+0000", 2024-09-16T11:43:55.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9309999999999997e-06 2024-09-16T11:43:55.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.370 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.370 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195737+0000", 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0719999999999999e-06 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195740+0000", 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1949999999999999e-06 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15639 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196469+0000", 2024-09-16T11:43:55.371 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00050388600000000005, 2024-09-16T11:43:55.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001937902, 2024-09-16T11:43:55.372 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.372 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.372 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.372 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.372 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.372 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15639 2024-09-16T11:43:55.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.372 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196469+0000", 2024-09-16T11:43:55.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196469+0000", 2024-09-16T11:43:55.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196469+0000", 2024-09-16T11:43:55.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.13e-07 2024-09-16T11:43:55.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196471+0000", 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6199999999999999e-06 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196476+0000", 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6239999999999999e-06 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196486+0000", 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5340000000000005e-06 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.374 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15672 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197102+0000", 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9998698, 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0013569000000000001, 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15672 2024-09-16T11:43:55.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.376 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197102+0000", 2024-09-16T11:43:55.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197102+0000", 2024-09-16T11:43:55.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197103+0000", 2024-09-16T11:43:55.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7000000000000003e-07 2024-09-16T11:43:55.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197104+0000", 2024-09-16T11:43:55.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1000000000000001e-06 2024-09-16T11:43:55.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197108+0000", 2024-09-16T11:43:55.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5279999999999999e-06 2024-09-16T11:43:55.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197110+0000", 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0449999999999999e-06 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16013 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.194415+0000", 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0025572080000000001, 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041011570000000002, 2024-09-16T11:43:55.378 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16013 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194415+0000", 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194415+0000", 2024-09-16T11:43:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194416+0000", 2024-09-16T11:43:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3799999999999998e-07 2024-09-16T11:43:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194418+0000", 2024-09-16T11:43:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.815e-06 2024-09-16T11:43:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194423+0000", 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194432+0000", 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.986e-06 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15599 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195759+0000", 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0012132950000000001, 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0028187619999999998, 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15599 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195759+0000", 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195759+0000", 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195760+0000", 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.06e-07 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195762+0000", 2024-09-16T11:43:55.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.973e-06 2024-09-16T11:43:55.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195768+0000", 2024-09-16T11:43:55.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6780000000000003e-06 2024-09-16T11:43:55.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195772+0000", 2024-09-16T11:43:55.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4649999999999999e-06 2024-09-16T11:43:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15640 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196489+0000", 2024-09-16T11:43:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.000483635, 2024-09-16T11:43:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021439050000000002, 2024-09-16T11:43:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.385 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15640 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196489+0000", 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196489+0000", 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196489+0000", 2024-09-16T11:43:55.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2899999999999999e-07 2024-09-16T11:43:55.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196491+0000", 2024-09-16T11:43:55.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.66e-06 2024-09-16T11:43:55.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196496+0000", 2024-09-16T11:43:55.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4330000000000003e-06 2024-09-16T11:43:55.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196499+0000", 2024-09-16T11:43:55.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.224e-06 2024-09-16T11:43:55.388 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.388 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.388 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.388 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15673 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.388 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197112+0000", 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9998603, 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001573893, 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15673 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197112+0000", 2024-09-16T11:43:55.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197112+0000", 2024-09-16T11:43:55.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197112+0000", 2024-09-16T11:43:55.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-07 2024-09-16T11:43:55.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197114+0000", 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1459999999999999e-06 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197116+0000", 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8169999999999999e-06 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197124+0000", 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1040000000000001e-06 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.391 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16014 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.194436+0000", 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0025366389999999998, 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0043054529999999999, 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16014 2024-09-16T11:43:55.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.393 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194436+0000", 2024-09-16T11:43:55.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194436+0000", 2024-09-16T11:43:55.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194436+0000", 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194438+0000", 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6950000000000001e-06 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194443+0000", 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5759999999999996e-06 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194451+0000", 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6199999999999999e-06 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15600 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195786+0000", 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001186084, 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0030068719999999998, 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.395 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15600 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195786+0000", 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195786+0000", 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195787+0000", 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4900000000000002e-07 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195788+0000", 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4270000000000001e-06 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195792+0000", 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0720000000000001e-06 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195795+0000", 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8480000000000001e-06 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15641 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.398 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196514+0000", 2024-09-16T11:43:55.399 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00045874499999999999, 2024-09-16T11:43:55.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0023322220000000001, 2024-09-16T11:43:55.399 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.399 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.399 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15641 2024-09-16T11:43:55.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.399 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196514+0000", 2024-09-16T11:43:55.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196514+0000", 2024-09-16T11:43:55.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196514+0000", 2024-09-16T11:43:55.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4000000000000002e-07 2024-09-16T11:43:55.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196516+0000", 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5799999999999999e-06 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196520+0000", 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.481e-06 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196524+0000", 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1439999999999996e-06 2024-09-16T11:43:55.401 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15674 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197126+0000", 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9998465, 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017763569999999999, 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.402 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15674 2024-09-16T11:43:55.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.403 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197126+0000", 2024-09-16T11:43:55.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197126+0000", 2024-09-16T11:43:55.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197126+0000", 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197127+0000", 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.026e-06 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197130+0000", 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8830000000000002e-06 2024-09-16T11:43:55.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197132+0000", 2024-09-16T11:43:55.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8390000000000001e-06 2024-09-16T11:43:55.405 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.405 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.405 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.405 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16015 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.405 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.194455+0000", 2024-09-16T11:43:55.405 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0025177429999999998, 2024-09-16T11:43:55.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0045008640000000003, 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16015 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194455+0000", 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194455+0000", 2024-09-16T11:43:55.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194455+0000", 2024-09-16T11:43:55.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:43:55.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194457+0000", 2024-09-16T11:43:55.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.562e-06 2024-09-16T11:43:55.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194462+0000", 2024-09-16T11:43:55.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2340000000000002e-06 2024-09-16T11:43:55.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194465+0000", 2024-09-16T11:43:55.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9960000000000002e-06 2024-09-16T11:43:55.408 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.408 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.408 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.409 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15601 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.409 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195807+0000", 2024-09-16T11:43:55.409 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001165976, 2024-09-16T11:43:55.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0032041359999999998, 2024-09-16T11:43:55.409 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.409 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.409 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.409 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.409 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.409 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15601 2024-09-16T11:43:55.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.409 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195807+0000", 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195807+0000", 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195807+0000", 2024-09-16T11:43:55.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.89e-07 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195808+0000", 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4869999999999999e-06 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195814+0000", 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5639999999999996e-06 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195817+0000", 2024-09-16T11:43:55.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9299999999999999e-06 2024-09-16T11:43:55.412 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.412 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.412 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.412 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15643 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.412 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196529+0000", 2024-09-16T11:43:55.412 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0004438, 2024-09-16T11:43:55.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025341740000000001, 2024-09-16T11:43:55.412 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.412 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.412 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.413 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.413 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.413 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15643 2024-09-16T11:43:55.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.413 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196529+0000", 2024-09-16T11:43:55.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196529+0000", 2024-09-16T11:43:55.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196529+0000", 2024-09-16T11:43:55.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0100000000000005e-07 2024-09-16T11:43:55.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196531+0000", 2024-09-16T11:43:55.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.595e-06 2024-09-16T11:43:55.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196536+0000", 2024-09-16T11:43:55.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1930000000000001e-06 2024-09-16T11:43:55.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196539+0000", 2024-09-16T11:43:55.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2459999999999999e-06 2024-09-16T11:43:55.415 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.415 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.415 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15675 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197134+0000", 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9998384, 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0019869509999999998, 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15675 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197134+0000", 2024-09-16T11:43:55.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197134+0000", 2024-09-16T11:43:55.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197134+0000", 2024-09-16T11:43:55.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.03e-07 2024-09-16T11:43:55.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197135+0000", 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3699999999999999e-07 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197138+0000", 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7669999999999999e-06 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197141+0000", 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3499999999999999e-06 2024-09-16T11:43:55.418 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16016 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.194468+0000", 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0025042829999999999, 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047041399999999999, 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.419 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16016 2024-09-16T11:43:55.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.420 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194468+0000", 2024-09-16T11:43:55.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194468+0000", 2024-09-16T11:43:55.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194468+0000", 2024-09-16T11:43:55.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:43:55.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194470+0000", 2024-09-16T11:43:55.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.598e-06 2024-09-16T11:43:55.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194476+0000", 2024-09-16T11:43:55.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.998e-06 2024-09-16T11:43:55.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.194479+0000", 2024-09-16T11:43:55.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9349999999999999e-06 2024-09-16T11:43:55.422 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.422 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.422 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.422 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15602 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.422 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195831+0000", 2024-09-16T11:43:55.422 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0011416149999999999, 2024-09-16T11:43:55.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0033878689999999999, 2024-09-16T11:43:55.423 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.423 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.423 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.423 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.423 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.423 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15602 2024-09-16T11:43:55.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.423 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195831+0000", 2024-09-16T11:43:55.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195831+0000", 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195831+0000", 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195833+0000", 2024-09-16T11:43:55.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.536e-06 2024-09-16T11:43:55.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195837+0000", 2024-09-16T11:43:55.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5170000000000001e-06 2024-09-16T11:43:55.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195841+0000", 2024-09-16T11:43:55.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3569999999999998e-06 2024-09-16T11:43:55.425 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.425 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15644 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196549+0000", 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.000423366, 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0026998320000000001, 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15644 2024-09-16T11:43:55.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.427 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196549+0000", 2024-09-16T11:43:55.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196549+0000", 2024-09-16T11:43:55.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196549+0000", 2024-09-16T11:43:55.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:43:55.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196568+0000", 2024-09-16T11:43:55.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8036000000000001e-05 2024-09-16T11:43:55.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196572+0000", 2024-09-16T11:43:55.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3120000000000004e-06 2024-09-16T11:43:55.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196574+0000", 2024-09-16T11:43:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.345e-06 2024-09-16T11:43:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15676 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197148+0000", 2024-09-16T11:43:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9998245, 2024-09-16T11:43:55.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021303369999999999, 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15676 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197148+0000", 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197148+0000", 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197148+0000", 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9400000000000001e-07 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197149+0000", 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.254e-06 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197152+0000", 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9229999999999998e-06 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197156+0000", 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1949999999999999e-06 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15603 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195848+0000", 2024-09-16T11:43:55.433 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0011249700000000001, 2024-09-16T11:43:55.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0034611910000000002, 2024-09-16T11:43:55.433 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.433 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.433 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.433 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.433 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.433 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15603 2024-09-16T11:43:55.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.433 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195848+0000", 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195848+0000", 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195848+0000", 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195849+0000", 2024-09-16T11:43:55.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6300000000000001e-06 2024-09-16T11:43:55.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195855+0000", 2024-09-16T11:43:55.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5709999999999997e-06 2024-09-16T11:43:55.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195863+0000", 2024-09-16T11:43:55.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2490000000000001e-06 2024-09-16T11:43:55.435 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.436 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.436 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.436 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15645 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.436 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196577+0000", 2024-09-16T11:43:55.436 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.000395778, 2024-09-16T11:43:55.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0027671190000000002, 2024-09-16T11:43:55.436 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.436 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.437 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15645 2024-09-16T11:43:55.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.437 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196577+0000", 2024-09-16T11:43:55.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196577+0000", 2024-09-16T11:43:55.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196577+0000", 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9900000000000002e-07 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196578+0000", 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2200000000000002e-07 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196582+0000", 2024-09-16T11:43:55.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196585+0000", 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7650000000000002e-06 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15677 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197163+0000", 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9998097, 2024-09-16T11:43:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022176489999999999, 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15677 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197163+0000", 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197163+0000", 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197163+0000", 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197164+0000", 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.249e-06 2024-09-16T11:43:55.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197167+0000", 2024-09-16T11:43:55.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9500000000000001e-06 2024-09-16T11:43:55.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197169+0000", 2024-09-16T11:43:55.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.889e-06 2024-09-16T11:43:55.442 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.442 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.442 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15608 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195867+0000", 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0011053790000000001, 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003565618, 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15608 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195867+0000", 2024-09-16T11:43:55.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195867+0000", 2024-09-16T11:43:55.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195867+0000", 2024-09-16T11:43:55.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2300000000000002e-07 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195869+0000", 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6959999999999999e-06 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195875+0000", 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.648e-06 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195877+0000", 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15646 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196600+0000", 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.000372658, 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029588240000000001, 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.446 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15646 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196600+0000", 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196600+0000", 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196600+0000", 2024-09-16T11:43:55.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:43:55.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196602+0000", 2024-09-16T11:43:55.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5689999999999999e-06 2024-09-16T11:43:55.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196608+0000", 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5350000000000001e-06 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196611+0000", 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.005e-06 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15678 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197176+0000", 2024-09-16T11:43:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9997969, 2024-09-16T11:43:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0024240220000000001, 2024-09-16T11:43:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15678 2024-09-16T11:43:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197176+0000", 2024-09-16T11:43:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197176+0000", 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197176+0000", 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0200000000000003e-07 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197177+0000", 2024-09-16T11:43:55.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2130000000000001e-06 2024-09-16T11:43:55.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197180+0000", 2024-09-16T11:43:55.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9529999999999999e-06 2024-09-16T11:43:55.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197182+0000", 2024-09-16T11:43:55.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9410000000000001e-06 2024-09-16T11:43:55.452 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.452 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.452 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15610 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195889+0000", 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0010837290000000001, 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037435749999999999, 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15610 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.453 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195889+0000", 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195889+0000", 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195889+0000", 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:43:55.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195891+0000", 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.567e-06 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195897+0000", 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7150000000000003e-06 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195900+0000", 2024-09-16T11:43:55.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0079999999999998e-06 2024-09-16T11:43:55.456 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.456 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.456 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.456 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15647 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.456 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196628+0000", 2024-09-16T11:43:55.456 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00034455, 2024-09-16T11:43:55.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003034592, 2024-09-16T11:43:55.456 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.456 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.456 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.456 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.457 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.457 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15647 2024-09-16T11:43:55.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.457 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196628+0000", 2024-09-16T11:43:55.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196628+0000", 2024-09-16T11:43:55.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196628+0000", 2024-09-16T11:43:55.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3099999999999999e-07 2024-09-16T11:43:55.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196629+0000", 2024-09-16T11:43:55.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3349999999999999e-06 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196634+0000", 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196636+0000", 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8380000000000002e-06 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.460 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15679 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.460 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197184+0000", 2024-09-16T11:43:55.460 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9997883, 2024-09-16T11:43:55.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025325370000000001, 2024-09-16T11:43:55.460 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.460 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.460 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.460 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.460 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.460 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15679 2024-09-16T11:43:55.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.460 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.461 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.461 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.461 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197184+0000", 2024-09-16T11:43:55.461 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.461 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.461 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.461 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197184+0000", 2024-09-16T11:43:55.461 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.461 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.461 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.461 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197184+0000", 2024-09-16T11:43:55.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.41e-07 2024-09-16T11:43:55.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.462 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197185+0000", 2024-09-16T11:43:55.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3699999999999999e-07 2024-09-16T11:43:55.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.462 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197188+0000", 2024-09-16T11:43:55.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7329999999999999e-06 2024-09-16T11:43:55.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197190+0000", 2024-09-16T11:43:55.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.84e-06 2024-09-16T11:43:55.463 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.463 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.463 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.463 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15611 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.463 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195916+0000", 2024-09-16T11:43:55.463 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0010568509999999999, 2024-09-16T11:43:55.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0038556799999999998, 2024-09-16T11:43:55.463 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15611 2024-09-16T11:43:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195916+0000", 2024-09-16T11:43:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195916+0000", 2024-09-16T11:43:55.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195916+0000", 2024-09-16T11:43:55.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5400000000000002e-07 2024-09-16T11:43:55.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195917+0000", 2024-09-16T11:43:55.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.393e-06 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195921+0000", 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1960000000000001e-06 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195931+0000", 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7270000000000002e-06 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.467 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15648 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.467 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196640+0000", 2024-09-16T11:43:55.467 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00033218800000000003, 2024-09-16T11:43:55.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031882590000000001, 2024-09-16T11:43:55.467 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.467 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.467 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.467 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.467 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.467 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15648 2024-09-16T11:43:55.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.467 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196640+0000", 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196640+0000", 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196641+0000", 2024-09-16T11:43:55.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4400000000000001e-07 2024-09-16T11:43:55.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196642+0000", 2024-09-16T11:43:55.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4360000000000001e-06 2024-09-16T11:43:55.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196648+0000", 2024-09-16T11:43:55.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3800000000000002e-06 2024-09-16T11:43:55.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196651+0000", 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0740000000000001e-06 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15680 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197198+0000", 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9997745, 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002671957, 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15680 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197198+0000", 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.471 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197198+0000", 2024-09-16T11:43:55.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197198+0000", 2024-09-16T11:43:55.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.96e-07 2024-09-16T11:43:55.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197202+0000", 2024-09-16T11:43:55.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9489999999999998e-06 2024-09-16T11:43:55.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197206+0000", 2024-09-16T11:43:55.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2710000000000001e-06 2024-09-16T11:43:55.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197208+0000", 2024-09-16T11:43:55.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0679999999999999e-06 2024-09-16T11:43:55.473 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.473 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.473 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15612 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195941+0000", 2024-09-16T11:43:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001031709, 2024-09-16T11:43:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039598690000000004, 2024-09-16T11:43:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15612 2024-09-16T11:43:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195941+0000", 2024-09-16T11:43:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195941+0000", 2024-09-16T11:43:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195941+0000", 2024-09-16T11:43:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:43:55.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195943+0000", 2024-09-16T11:43:55.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7829999999999999e-06 2024-09-16T11:43:55.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195949+0000", 2024-09-16T11:43:55.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6489999999999996e-06 2024-09-16T11:43:55.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195952+0000", 2024-09-16T11:43:55.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6049999999999998e-06 2024-09-16T11:43:55.477 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.477 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.477 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.477 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15649 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.477 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196654+0000", 2024-09-16T11:43:55.477 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.000318583, 2024-09-16T11:43:55.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0032734069999999999, 2024-09-16T11:43:55.477 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15649 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196654+0000", 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196654+0000", 2024-09-16T11:43:55.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196654+0000", 2024-09-16T11:43:55.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:43:55.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196656+0000", 2024-09-16T11:43:55.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5090000000000001e-06 2024-09-16T11:43:55.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196661+0000", 2024-09-16T11:43:55.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3639999999999998e-06 2024-09-16T11:43:55.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196669+0000", 2024-09-16T11:43:55.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1200000000000002e-06 2024-09-16T11:43:55.480 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.480 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.480 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.481 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15681 4.7 4.55c718a7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.481 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197210+0000", 2024-09-16T11:43:55.481 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9997625, 2024-09-16T11:43:55.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0027620069999999999, 2024-09-16T11:43:55.481 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.481 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.481 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.481 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.481 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.481 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15681 2024-09-16T11:43:55.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.482 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197210+0000", 2024-09-16T11:43:55.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197210+0000", 2024-09-16T11:43:55.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197210+0000", 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.53e-07 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197213+0000", 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9660000000000001e-06 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197217+0000", 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4010000000000001e-06 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197219+0000", 2024-09-16T11:43:55.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.649e-06 2024-09-16T11:43:55.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.484 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.484 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15613 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.484 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195956+0000", 2024-09-16T11:43:55.484 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001016695, 2024-09-16T11:43:55.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0040693500000000002, 2024-09-16T11:43:55.484 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.484 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.485 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15613 2024-09-16T11:43:55.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.485 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195956+0000", 2024-09-16T11:43:55.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195956+0000", 2024-09-16T11:43:55.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195956+0000", 2024-09-16T11:43:55.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:43:55.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195958+0000", 2024-09-16T11:43:55.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5200000000000001e-06 2024-09-16T11:43:55.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195963+0000", 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.169e-06 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195965+0000", 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.605e-06 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.487 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15650 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.488 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196673+0000", 2024-09-16T11:43:55.488 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00029993199999999999, 2024-09-16T11:43:55.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0034051569999999998, 2024-09-16T11:43:55.488 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.488 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.488 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.488 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.488 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.488 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15650 2024-09-16T11:43:55.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.488 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196673+0000", 2024-09-16T11:43:55.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196673+0000", 2024-09-16T11:43:55.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196673+0000", 2024-09-16T11:43:55.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.39e-07 2024-09-16T11:43:55.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196674+0000", 2024-09-16T11:43:55.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.598e-06 2024-09-16T11:43:55.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196680+0000", 2024-09-16T11:43:55.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5010000000000004e-06 2024-09-16T11:43:55.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196683+0000", 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.531e-06 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15682 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197221+0000", 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9997511, 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029173459999999999, 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.491 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15682 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197221+0000", 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197221+0000", 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197222+0000", 2024-09-16T11:43:55.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:43:55.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197223+0000", 2024-09-16T11:43:55.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0599999999999999e-07 2024-09-16T11:43:55.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197225+0000", 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.694e-06 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197227+0000", 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8840000000000001e-06 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15614 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.195977+0000", 2024-09-16T11:43:55.494 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00099500200000000004, 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00419338, 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15614 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195977+0000", 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195977+0000", 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195978+0000", 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1600000000000001e-07 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195980+0000", 2024-09-16T11:43:55.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5120000000000001e-06 2024-09-16T11:43:55.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195985+0000", 2024-09-16T11:43:55.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6529999999999999e-06 2024-09-16T11:43:55.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.195988+0000", 2024-09-16T11:43:55.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9409999999999999e-06 2024-09-16T11:43:55.497 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.497 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15651 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196685+0000", 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00028715499999999998, 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003514727, 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15651 2024-09-16T11:43:55.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.499 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196685+0000", 2024-09-16T11:43:55.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196685+0000", 2024-09-16T11:43:55.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196686+0000", 2024-09-16T11:43:55.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8599999999999998e-07 2024-09-16T11:43:55.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196687+0000", 2024-09-16T11:43:55.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.457e-06 2024-09-16T11:43:55.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196693+0000", 2024-09-16T11:43:55.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7930000000000003e-06 2024-09-16T11:43:55.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196696+0000", 2024-09-16T11:43:55.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1039999999999998e-06 2024-09-16T11:43:55.501 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.501 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.501 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.501 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15683 4.7 4.55c718a7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.501 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197235+0000", 2024-09-16T11:43:55.501 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9997373, 2024-09-16T11:43:55.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029929549999999998, 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15683 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197235+0000", 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197235+0000", 2024-09-16T11:43:55.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197236+0000", 2024-09-16T11:43:55.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:43:55.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197237+0000", 2024-09-16T11:43:55.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7900000000000007e-07 2024-09-16T11:43:55.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197241+0000", 2024-09-16T11:43:55.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8609999999999997e-06 2024-09-16T11:43:55.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197245+0000", 2024-09-16T11:43:55.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.151e-06 2024-09-16T11:43:55.504 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.504 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.504 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.505 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15615 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.505 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196009+0000", 2024-09-16T11:43:55.505 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00096393200000000003, 2024-09-16T11:43:55.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0042733459999999999, 2024-09-16T11:43:55.505 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.505 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.505 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.505 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.505 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.505 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15615 2024-09-16T11:43:55.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.506 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196009+0000", 2024-09-16T11:43:55.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196009+0000", 2024-09-16T11:43:55.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196009+0000", 2024-09-16T11:43:55.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:43:55.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196018+0000", 2024-09-16T11:43:55.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8380000000000008e-06 2024-09-16T11:43:55.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196023+0000", 2024-09-16T11:43:55.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3990000000000003e-06 2024-09-16T11:43:55.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196027+0000", 2024-09-16T11:43:55.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7400000000000002e-06 2024-09-16T11:43:55.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.508 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.508 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15652 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.508 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196700+0000", 2024-09-16T11:43:55.508 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00027295299999999999, 2024-09-16T11:43:55.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0036350110000000001, 2024-09-16T11:43:55.509 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.509 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.509 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.509 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.509 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.509 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15652 2024-09-16T11:43:55.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.509 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196700+0000", 2024-09-16T11:43:55.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196700+0000", 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196700+0000", 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0100000000000001e-07 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196701+0000", 2024-09-16T11:43:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5060000000000001e-06 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196707+0000", 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3800000000000002e-06 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196710+0000", 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0110000000000001e-06 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15684 4.7 4.55c718a7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197270+0000", 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.999702, 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031202479999999999, 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15684 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197270+0000", 2024-09-16T11:43:55.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197270+0000", 2024-09-16T11:43:55.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197271+0000", 2024-09-16T11:43:55.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:43:55.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197273+0000", 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8220000000000001e-06 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197278+0000", 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2950000000000001e-06 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197281+0000", 2024-09-16T11:43:55.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4959999999999999e-06 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15616 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196046+0000", 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00092616200000000001, 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0043925420000000001, 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.515 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.516 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15616 2024-09-16T11:43:55.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.516 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196046+0000", 2024-09-16T11:43:55.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196046+0000", 2024-09-16T11:43:55.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196047+0000", 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.01e-07 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196048+0000", 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.685e-06 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196053+0000", 2024-09-16T11:43:55.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8629999999999999e-06 2024-09-16T11:43:55.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196057+0000", 2024-09-16T11:43:55.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0010000000000003e-06 2024-09-16T11:43:55.518 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.518 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.518 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.518 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15653 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.518 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196725+0000", 2024-09-16T11:43:55.518 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00024708599999999999, 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037430979999999998, 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15653 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196725+0000", 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196725+0000", 2024-09-16T11:43:55.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196726+0000", 2024-09-16T11:43:55.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:43:55.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196727+0000", 2024-09-16T11:43:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.26e-06 2024-09-16T11:43:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196731+0000", 2024-09-16T11:43:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.104e-06 2024-09-16T11:43:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196734+0000", 2024-09-16T11:43:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9249999999999999e-06 2024-09-16T11:43:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.521 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15685 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197302+0000", 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.99967, 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00319455, 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15685 2024-09-16T11:43:55.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197302+0000", 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197302+0000", 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197303+0000", 2024-09-16T11:43:55.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2599999999999998e-07 2024-09-16T11:43:55.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197305+0000", 2024-09-16T11:43:55.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.767e-06 2024-09-16T11:43:55.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197310+0000", 2024-09-16T11:43:55.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5690000000000004e-06 2024-09-16T11:43:55.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197317+0000", 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0419999999999997e-06 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15617 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196074+0000", 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00089890499999999997, 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0044754520000000004, 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.525 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15617 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196074+0000", 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196074+0000", 2024-09-16T11:43:55.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196074+0000", 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2800000000000003e-07 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196076+0000", 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7120000000000001e-06 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196080+0000", 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6789999999999996e-06 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196089+0000", 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5099999999999998e-06 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15654 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196745+0000", 2024-09-16T11:43:55.528 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00022717000000000001, 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0038565439999999999, 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15654 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196745+0000", 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196745+0000", 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196746+0000", 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196747+0000", 2024-09-16T11:43:55.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4759999999999999e-06 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196753+0000", 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6189999999999999e-06 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196760+0000", 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5140000000000004e-06 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.532 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15686 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.532 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197319+0000", 2024-09-16T11:43:55.532 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9996529, 2024-09-16T11:43:55.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0033354980000000001, 2024-09-16T11:43:55.532 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.532 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.532 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.532 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.532 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.532 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15686 2024-09-16T11:43:55.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.532 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197319+0000", 2024-09-16T11:43:55.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197319+0000", 2024-09-16T11:43:55.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197320+0000", 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6e-07 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197321+0000", 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197325+0000", 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.827e-06 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197329+0000", 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3769999999999998e-06 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15619 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196092+0000", 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00087998199999999996, 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046040439999999998, 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.535 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.536 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15619 2024-09-16T11:43:55.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.536 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.536 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.536 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196092+0000", 2024-09-16T11:43:55.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.536 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.536 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196092+0000", 2024-09-16T11:43:55.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196093+0000", 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9999999999999998e-07 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196099+0000", 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6829999999999996e-06 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196105+0000", 2024-09-16T11:43:55.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9150000000000001e-06 2024-09-16T11:43:55.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196108+0000", 2024-09-16T11:43:55.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3969999999999998e-06 2024-09-16T11:43:55.538 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.538 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.538 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.538 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15655 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.538 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196783+0000", 2024-09-16T11:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00018903299999999999, 2024-09-16T11:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039398899999999997, 2024-09-16T11:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15655 2024-09-16T11:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196783+0000", 2024-09-16T11:43:55.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196783+0000", 2024-09-16T11:43:55.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196784+0000", 2024-09-16T11:43:55.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:43:55.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196786+0000", 2024-09-16T11:43:55.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7689999999999999e-06 2024-09-16T11:43:55.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196791+0000", 2024-09-16T11:43:55.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5999999999999997e-06 2024-09-16T11:43:55.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196794+0000", 2024-09-16T11:43:55.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8339999999999999e-06 2024-09-16T11:43:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15687 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197331+0000", 2024-09-16T11:43:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9996409, 2024-09-16T11:43:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0034210500000000001, 2024-09-16T11:43:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15687 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197331+0000", 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197331+0000", 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197332+0000", 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.75e-07 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197333+0000", 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.407e-06 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197339+0000", 2024-09-16T11:43:55.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7130000000000002e-06 2024-09-16T11:43:55.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197342+0000", 2024-09-16T11:43:55.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0979999999999998e-06 2024-09-16T11:43:55.545 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.545 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.545 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.545 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15620 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.545 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196113+0000", 2024-09-16T11:43:55.545 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.000859733, 2024-09-16T11:43:55.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046912450000000001, 2024-09-16T11:43:55.546 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.546 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.546 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.546 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.546 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.546 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15620 2024-09-16T11:43:55.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.546 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196113+0000", 2024-09-16T11:43:55.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196113+0000", 2024-09-16T11:43:55.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196113+0000", 2024-09-16T11:43:55.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:43:55.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196115+0000", 2024-09-16T11:43:55.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.649e-06 2024-09-16T11:43:55.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196119+0000", 2024-09-16T11:43:55.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3800000000000004e-06 2024-09-16T11:43:55.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196122+0000", 2024-09-16T11:43:55.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.128e-06 2024-09-16T11:43:55.548 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.549 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.549 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.549 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15656 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.549 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196800+0000", 2024-09-16T11:43:55.549 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00017295599999999999, 2024-09-16T11:43:55.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0040595780000000003, 2024-09-16T11:43:55.549 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.549 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.549 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.549 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.549 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15656 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196800+0000", 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196800+0000", 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196800+0000", 2024-09-16T11:43:55.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.04e-07 2024-09-16T11:43:55.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196808+0000", 2024-09-16T11:43:55.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8569999999999995e-06 2024-09-16T11:43:55.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196814+0000", 2024-09-16T11:43:55.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9610000000000002e-06 2024-09-16T11:43:55.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196818+0000", 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1869999999999999e-06 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15688 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197346+0000", 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9996266, 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0035658069999999998, 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.552 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.553 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.553 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.553 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.553 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15688 2024-09-16T11:43:55.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.553 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197346+0000", 2024-09-16T11:43:55.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197346+0000", 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197346+0000", 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197348+0000", 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.497e-06 2024-09-16T11:43:55.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197353+0000", 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4280000000000004e-06 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197356+0000", 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9560000000000002e-06 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.555 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15621 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.556 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196142+0000", 2024-09-16T11:43:55.556 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00083090599999999996, 2024-09-16T11:43:55.556 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048121839999999997, 2024-09-16T11:43:55.556 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.556 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.556 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15621 2024-09-16T11:43:55.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.556 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.556 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196142+0000", 2024-09-16T11:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196142+0000", 2024-09-16T11:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196142+0000", 2024-09-16T11:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9999999999999998e-07 2024-09-16T11:43:55.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196144+0000", 2024-09-16T11:43:55.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6479999999999999e-06 2024-09-16T11:43:55.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196150+0000", 2024-09-16T11:43:55.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1879999999999999e-06 2024-09-16T11:43:55.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196153+0000", 2024-09-16T11:43:55.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0240000000000002e-06 2024-09-16T11:43:55.559 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.559 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.559 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.559 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15657 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.559 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196822+0000", 2024-09-16T11:43:55.559 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00015009800000000001, 2024-09-16T11:43:55.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041591759999999997, 2024-09-16T11:43:55.560 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.560 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.560 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.560 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.560 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.560 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15657 2024-09-16T11:43:55.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.560 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.560 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196822+0000", 2024-09-16T11:43:55.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.561 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196822+0000", 2024-09-16T11:43:55.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.561 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196823+0000", 2024-09-16T11:43:55.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8599999999999998e-07 2024-09-16T11:43:55.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196824+0000", 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.539e-06 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196830+0000", 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3700000000000003e-06 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196833+0000", 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0649999999999999e-06 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.562 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.563 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.563 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15689 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.563 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197384+0000", 2024-09-16T11:43:55.563 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.999589, 2024-09-16T11:43:55.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003640782, 2024-09-16T11:43:55.563 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.563 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.563 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.563 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.563 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.563 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15689 2024-09-16T11:43:55.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.564 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197384+0000", 2024-09-16T11:43:55.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197384+0000", 2024-09-16T11:43:55.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197384+0000", 2024-09-16T11:43:55.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.65e-07 2024-09-16T11:43:55.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197385+0000", 2024-09-16T11:43:55.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:43:55.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197391+0000", 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3859999999999998e-06 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197394+0000", 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.269e-06 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15622 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196170+0000", 2024-09-16T11:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00080268699999999995, 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0049075309999999997, 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15622 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196170+0000", 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196170+0000", 2024-09-16T11:43:55.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196170+0000", 2024-09-16T11:43:55.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:43:55.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196172+0000", 2024-09-16T11:43:55.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7060000000000001e-06 2024-09-16T11:43:55.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196177+0000", 2024-09-16T11:43:55.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5049999999999998e-06 2024-09-16T11:43:55.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196181+0000", 2024-09-16T11:43:55.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1879999999999998e-06 2024-09-16T11:43:55.569 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.569 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15658 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196836+0000", 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00013646, 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0043012520000000002, 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15658 2024-09-16T11:43:55.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196836+0000", 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196836+0000", 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196836+0000", 2024-09-16T11:43:55.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:43:55.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.572 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.572 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196838+0000", 2024-09-16T11:43:55.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3999999999999999e-06 2024-09-16T11:43:55.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.572 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.572 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196842+0000", 2024-09-16T11:43:55.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9249999999999997e-06 2024-09-16T11:43:55.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196850+0000", 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9880000000000004e-06 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15690 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197410+0000", 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9995623, 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037745449999999998, 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.573 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15690 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197410+0000", 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197410+0000", 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197411+0000", 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2300000000000002e-07 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197412+0000", 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7290000000000001e-06 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197418+0000", 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4720000000000003e-06 2024-09-16T11:43:55.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.576 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197421+0000", 2024-09-16T11:43:55.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7989999999999998e-06 2024-09-16T11:43:55.576 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.576 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.576 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.576 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15623 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.576 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196191+0000", 2024-09-16T11:43:55.576 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00078165999999999999, 2024-09-16T11:43:55.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0050225290000000004, 2024-09-16T11:43:55.577 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.577 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.577 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.577 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.578 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.578 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15623 2024-09-16T11:43:55.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.578 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196191+0000", 2024-09-16T11:43:55.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196191+0000", 2024-09-16T11:43:55.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196191+0000", 2024-09-16T11:43:55.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0200000000000002e-07 2024-09-16T11:43:55.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196193+0000", 2024-09-16T11:43:55.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.61e-06 2024-09-16T11:43:55.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196199+0000", 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0499999999999997e-06 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196202+0000", 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9969999999999999e-06 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15659 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.580 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196854+0000", 2024-09-16T11:43:55.581 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.000118799, 2024-09-16T11:43:55.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0043907379999999999, 2024-09-16T11:43:55.581 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.581 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.581 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.581 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.581 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.581 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15659 2024-09-16T11:43:55.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.581 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.581 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196854+0000", 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196854+0000", 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196854+0000", 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6600000000000002e-07 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196856+0000", 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6390000000000001e-06 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196861+0000", 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6550000000000001e-06 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196865+0000", 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1410000000000001e-06 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.583 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.584 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.584 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15691 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.584 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197427+0000", 2024-09-16T11:43:55.584 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9995456, 2024-09-16T11:43:55.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003863572, 2024-09-16T11:43:55.584 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.584 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.584 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.584 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.584 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.584 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15691 2024-09-16T11:43:55.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.585 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.585 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197427+0000", 2024-09-16T11:43:55.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.585 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197427+0000", 2024-09-16T11:43:55.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197427+0000", 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197429+0000", 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.297e-06 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197433+0000", 2024-09-16T11:43:55.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0029999999999996e-06 2024-09-16T11:43:55.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.587 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197435+0000", 2024-09-16T11:43:55.587 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8499999999999998e-06 2024-09-16T11:43:55.587 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.587 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.587 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.587 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15625 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.587 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196218+0000", 2024-09-16T11:43:55.587 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00075469800000000004, 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.005129191, 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15625 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196218+0000", 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196218+0000", 2024-09-16T11:43:55.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196218+0000", 2024-09-16T11:43:55.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2700000000000001e-07 2024-09-16T11:43:55.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196219+0000", 2024-09-16T11:43:55.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4619999999999999e-06 2024-09-16T11:43:55.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196224+0000", 2024-09-16T11:43:55.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0879999999999997e-06 2024-09-16T11:43:55.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196226+0000", 2024-09-16T11:43:55.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8839999999999998e-06 2024-09-16T11:43:55.590 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.590 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15660 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196868+0000", 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00010458699999999999, 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0045315399999999997, 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15660 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.591 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.592 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.592 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196868+0000", 2024-09-16T11:43:55.592 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.592 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.592 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.592 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196868+0000", 2024-09-16T11:43:55.592 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.592 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.592 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196868+0000", 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196870+0000", 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.454e-06 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196875+0000", 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4399999999999996e-06 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196878+0000", 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9840000000000001e-06 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15692 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197447+0000", 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9995251, 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003995218, 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.594 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15692 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197447+0000", 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197447+0000", 2024-09-16T11:43:55.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197448+0000", 2024-09-16T11:43:55.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9900000000000001e-07 2024-09-16T11:43:55.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197454+0000", 2024-09-16T11:43:55.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4590000000000003e-06 2024-09-16T11:43:55.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197459+0000", 2024-09-16T11:43:55.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.079e-06 2024-09-16T11:43:55.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197467+0000", 2024-09-16T11:43:55.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6450000000000005e-06 2024-09-16T11:43:55.597 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.597 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.597 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15626 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196238+0000", 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00073445200000000002, 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0052327440000000001, 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15626 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.598 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196238+0000", 2024-09-16T11:43:55.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.599 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196238+0000", 2024-09-16T11:43:55.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.599 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196239+0000", 2024-09-16T11:43:55.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:43:55.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196240+0000", 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5060000000000001e-06 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196246+0000", 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196249+0000", 2024-09-16T11:43:55.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9780000000000001e-06 2024-09-16T11:43:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15661 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196894+0000", 2024-09-16T11:43:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 7.8603999999999995e-05, 2024-09-16T11:43:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046140230000000001, 2024-09-16T11:43:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15661 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196894+0000", 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196894+0000", 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196894+0000", 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196895+0000", 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.361e-06 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196899+0000", 2024-09-16T11:43:55.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0319999999999997e-06 2024-09-16T11:43:55.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196902+0000", 2024-09-16T11:43:55.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9469999999999999e-06 2024-09-16T11:43:55.604 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.604 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.604 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.604 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15693 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.604 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197472+0000", 2024-09-16T11:43:55.604 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9995008, 2024-09-16T11:43:55.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0040880320000000001, 2024-09-16T11:43:55.605 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.605 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.605 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.605 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.605 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.605 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15693 2024-09-16T11:43:55.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.605 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.605 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.605 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197472+0000", 2024-09-16T11:43:55.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197472+0000", 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197472+0000", 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2799999999999996e-07 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197477+0000", 2024-09-16T11:43:55.606 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4610000000000001e-06 2024-09-16T11:43:55.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.607 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.607 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197482+0000", 2024-09-16T11:43:55.607 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6200000000000004e-06 2024-09-16T11:43:55.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.607 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.607 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197486+0000", 2024-09-16T11:43:55.607 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3689999999999998e-06 2024-09-16T11:43:55.607 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.607 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.607 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.608 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15627 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.608 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196265+0000", 2024-09-16T11:43:55.608 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00070787300000000003, 2024-09-16T11:43:55.608 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0053527790000000002, 2024-09-16T11:43:55.608 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.608 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.608 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.608 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.608 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.608 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15627 2024-09-16T11:43:55.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196265+0000", 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196265+0000", 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.609 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196265+0000", 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5400000000000002e-07 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196266+0000", 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.401e-06 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196270+0000", 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9550000000000002e-06 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.610 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196273+0000", 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.066e-06 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15662 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196914+0000", 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 5.8415999999999998e-05, 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047561900000000004, 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.611 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.612 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.612 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.612 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15662 2024-09-16T11:43:55.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.612 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.612 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196914+0000", 2024-09-16T11:43:55.612 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.612 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196914+0000", 2024-09-16T11:43:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196915+0000", 2024-09-16T11:43:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.75e-07 2024-09-16T11:43:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196916+0000", 2024-09-16T11:43:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.454e-06 2024-09-16T11:43:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.614 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.614 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196922+0000", 2024-09-16T11:43:55.614 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.57e-06 2024-09-16T11:43:55.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.614 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.614 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.614 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196925+0000", 2024-09-16T11:43:55.614 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9390000000000002e-06 2024-09-16T11:43:55.614 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.614 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.614 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.614 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15694 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197489+0000", 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9994831, 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0042129250000000002, 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15694 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197489+0000", 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197489+0000", 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197490+0000", 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197491+0000", 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.528e-06 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197496+0000", 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.622e-06 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197504+0000", 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8879999999999997e-06 2024-09-16T11:43:55.617 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.618 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.618 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.618 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15629 4.7 4.dc38827 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.618 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196285+0000", 2024-09-16T11:43:55.618 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00068769299999999996, 2024-09-16T11:43:55.618 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0054453169999999999, 2024-09-16T11:43:55.618 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.618 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.618 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.618 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.618 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15629 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196285+0000", 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196285+0000", 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.620 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.620 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196285+0000", 2024-09-16T11:43:55.620 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1500000000000005e-07 2024-09-16T11:43:55.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.620 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.620 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196287+0000", 2024-09-16T11:43:55.620 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.55e-06 2024-09-16T11:43:55.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.620 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.620 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196292+0000", 2024-09-16T11:43:55.620 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5180000000000004e-06 2024-09-16T11:43:55.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.621 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.621 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196301+0000", 2024-09-16T11:43:55.621 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6149999999999997e-06 2024-09-16T11:43:55.621 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.621 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.621 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.621 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15663 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.621 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.196940+0000", 2024-09-16T11:43:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.2042999999999998e-05, 2024-09-16T11:43:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048316499999999998, 2024-09-16T11:43:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15663 2024-09-16T11:43:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196940+0000", 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196940+0000", 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196941+0000", 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2100000000000001e-07 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.623 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196942+0000", 2024-09-16T11:43:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2699999999999999e-06 2024-09-16T11:43:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196946+0000", 2024-09-16T11:43:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9809999999999997e-06 2024-09-16T11:43:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.196949+0000", 2024-09-16T11:43:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2720000000000002e-06 2024-09-16T11:43:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:15695 4.7 4.96df947 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.197507+0000", 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9994655, 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0043178640000000003, 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.625 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 15695 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197507+0000", 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197507+0000", 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.626 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.627 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197508+0000", 2024-09-16T11:43:55.627 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.06e-07 2024-09-16T11:43:55.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.627 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.627 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197509+0000", 2024-09-16T11:43:55.627 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.59e-06 2024-09-16T11:43:55.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.627 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.627 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197515+0000", 2024-09-16T11:43:55.627 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5979999999999996e-06 2024-09-16T11:43:55.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.628 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.628 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.628 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.197518+0000", 2024-09-16T11:43:55.628 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1779999999999999e-06 2024-09-16T11:43:55.628 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.628 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.628 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.628 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.628 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:43:55.628 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 123 2024-09-16T11:43:55.628 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:55.629 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:43:55.629 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:43:55.630 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:43:55.631 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:43:55.631 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:43:55.633 INFO:teuthology.orchestra.run.smithi070.stdout:2798 write 0x4b9bd7a thru 0x4ba6b73 (0xadfa bytes) 2024-09-16T11:43:55.633 INFO:teuthology.orchestra.run.smithi070.stdout:2799 punch from 0x2212455 to 0x2220114, (0xdcbf bytes) 2024-09-16T11:43:55.633 INFO:teuthology.orchestra.run.smithi070.stdout:2800 write 0xc1d59a thru 0xc1ecac (0x1713 bytes) 2024-09-16T11:43:55.633 INFO:teuthology.orchestra.run.smithi070.stdout:2801 read 0x28eb723 thru 0x28f0ddc (0x56ba bytes) 2024-09-16T11:43:55.633 INFO:teuthology.orchestra.run.smithi070.stdout:2802 trunc from 0xe9e863f to 0x511b84d 2024-09-16T11:43:55.633 INFO:teuthology.orchestra.run.smithi070.stdout:2803 trunc from 0x511b84d to 0xa29db35 2024-09-16T11:43:55.633 INFO:teuthology.orchestra.run.smithi070.stdout:2804 write 0x112cf thru 0x1d7e1 (0xc513 bytes) 2024-09-16T11:43:55.633 INFO:teuthology.orchestra.run.smithi070.stdout:2806 trunc from 0xa29db35 to 0x1e306db 2024-09-16T11:43:55.633 INFO:teuthology.orchestra.run.smithi070.stdout:2807 trunc from 0x1e306db to 0x30538ae 2024-09-16T11:43:55.634 INFO:teuthology.orchestra.run.smithi070.stdout:2809 writesame 0x2aa0cdb thru 0x2aafde2 data_size 0x24(0xf108 bytes) 2024-09-16T11:43:55.634 INFO:teuthology.orchestra.run.smithi070.stdout:2810 read 0x1c7542d thru 0x1c7e1ad (0x8d81 bytes) 2024-09-16T11:43:55.634 INFO:teuthology.orchestra.run.smithi070.stdout:2811 trunc from 0x30538ae to 0x9d0cbaa 2024-09-16T11:43:55.634 INFO:teuthology.orchestra.run.smithi070.stdout:2812 write 0x72c12f1 thru 0x72c7ca0 (0x69b0 bytes) 2024-09-16T11:43:55.634 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:55.627+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:55.639 INFO:teuthology.orchestra.run.smithi070.stdout:2813 write 0xcf79df2 thru 0xcf83ecf (0xa0de bytes) 2024-09-16T11:43:55.647 INFO:teuthology.orchestra.run.smithi070.stdout:2816 read 0x7303165 thru 0x7304929 (0x17c5 bytes) 2024-09-16T11:43:55.647 INFO:teuthology.orchestra.run.smithi070.stdout:2818 read 0x86c4c14 thru 0x86c66aa (0x1a97 bytes) 2024-09-16T11:43:55.647 INFO:teuthology.orchestra.run.smithi070.stdout:2820 punch from 0x89c6fea to 0x89c9e67, (0x2e7d bytes) 2024-09-16T11:43:55.647 INFO:teuthology.orchestra.run.smithi070.stdout:2822 write 0xb45d47a thru 0xb464137 (0x6cbe bytes) 2024-09-16T11:43:55.657 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:55.673 INFO:teuthology.orchestra.run.smithi070.stdout:2824 trunc from 0xcf83ed0 to 0x5f9b6a 2024-09-16T11:43:55.722 INFO:teuthology.orchestra.run.smithi070.stdout:2825 trunc from 0x5f9b6a to 0x1d6db7 2024-09-16T11:43:55.731 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:55.728+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:55.739 INFO:teuthology.orchestra.run.smithi070.stdout:2826 trunc from 0x1d6db7 to 0x713abf9 2024-09-16T11:43:55.744 INFO:teuthology.orchestra.run.smithi070.stdout:2827 read 0x6388c29 thru 0x6396d67 (0xe13f bytes) 2024-09-16T11:43:55.744 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:55.744 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:55.744 INFO:teuthology.orchestra.run.smithi070.stdout:2832 read 0x51f6dc2 thru 0x51fe092 (0x72d1 bytes) 2024-09-16T11:43:55.750 INFO:teuthology.orchestra.run.smithi070.stdout:2834 writesame 0x80ecee4 thru 0x80f8f98 data_size 0x1(0xc0b5 bytes) 2024-09-16T11:43:55.750 INFO:teuthology.orchestra.run.smithi070.stdout:2835 writesame 0x80232b6 thru 0x802c984 data_size 0x11(0x96cf bytes) 2024-09-16T11:43:55.759 INFO:teuthology.orchestra.run.smithi070.stdout:2836 clone 14 order 18 su 16384 sc 2 2024-09-16T11:43:55.766 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:55.767 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:55.767 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:55.767 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:55.767 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:55.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:43:55.831 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:55.829+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:55.925 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:55.925 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:55.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:55.925 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19182 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3932160~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.507225+0000", 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.415388437, 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048909983999999997, 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19182 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.507225+0000", 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.507225+0000", 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.509102+0000", 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001877213 2024-09-16T11:43:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.509107+0000", 2024-09-16T11:43:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1950000000000002e-06 2024-09-16T11:43:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.509117+0000", 2024-09-16T11:43:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8619999999999998e-06 2024-09-16T11:43:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.509122+0000", 2024-09-16T11:43:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:43:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.556023+0000", 2024-09-16T11:43:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.046900305000000003 2024-09-16T11:43:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.556064+0000", 2024-09-16T11:43:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1859e-05 2024-09-16T11:43:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.556135+0000", 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0241999999999996e-05 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19364 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1171456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40933530600000001, 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049001805000000002, 2024-09-16T11:43:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19364 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:43:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:43:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:43:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513280+0000", 2024-09-16T11:43:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7090000000000001e-06 2024-09-16T11:43:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513286+0000", 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5840000000000002e-06 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513290+0000", 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3680000000000004e-06 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562121+0000", 2024-09-16T11:43:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048831324000000002 2024-09-16T11:43:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562195+0000", 2024-09-16T11:43:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3415000000000001e-05 2024-09-16T11:43:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562280+0000", 2024-09-16T11:43:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4975999999999994e-05 2024-09-16T11:43:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19365 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1490944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40931556699999999, 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049335349000000001, 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19365 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513300+0000", 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3020000000000002e-06 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513306+0000", 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8780000000000003e-06 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513310+0000", 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1309999999999999e-06 2024-09-16T11:43:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562322+0000", 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049011635999999997 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562544+0000", 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022168000000000001 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562633+0000", 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9350000000000003e-05 2024-09-16T11:43:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19366 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1916928~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40929863500000002, 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049865073000000003, 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19366 2024-09-16T11:43:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:43:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:43:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515102+0000", 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017877419999999999 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515106+0000", 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4450000000000001e-06 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515113+0000", 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7680000000000008e-06 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515117+0000", 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562756+0000", 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047638579 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563093+0000", 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000337516 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563180+0000", 2024-09-16T11:43:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6420000000000003e-05 2024-09-16T11:43:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19371 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3194880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:43:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40740142800000001, 2024-09-16T11:43:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048948356999999998, 2024-09-16T11:43:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19371 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515214+0000", 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5969999999999999e-06 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515219+0000", 2024-09-16T11:43:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4559999999999999e-06 2024-09-16T11:43:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515223+0000", 2024-09-16T11:43:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:43:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563991+0000", 2024-09-16T11:43:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048768911999999998 2024-09-16T11:43:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564063+0000", 2024-09-16T11:43:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2010999999999999e-05 2024-09-16T11:43:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564160+0000", 2024-09-16T11:43:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6617e-05 2024-09-16T11:43:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19372 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3620864~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:43:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40738721100000003, 2024-09-16T11:43:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049130502, 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19372 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515228+0000", 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515233+0000", 2024-09-16T11:43:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:43:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515237+0000", 2024-09-16T11:43:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.27e-06 2024-09-16T11:43:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564199+0000", 2024-09-16T11:43:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048961787999999999 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564275+0000", 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6001000000000006e-05 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564356+0000", 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1941999999999996e-05 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19373 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3940352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.407372811, 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049289588000000002, 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19373 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:43:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:43:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515241+0000", 2024-09-16T11:43:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515242+0000", 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.528e-06 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515248+0000", 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515251+0000", 2024-09-16T11:43:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2480000000000001e-06 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564388+0000", 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049137313000000002 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564455+0000", 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7049000000000001e-05 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564530+0000", 2024-09-16T11:43:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4626e-05 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19374 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 4046848~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40735840800000001, 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049457133, 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19374 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:43:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:43:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515257+0000", 2024-09-16T11:43:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5740000000000001e-06 2024-09-16T11:43:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515262+0000", 2024-09-16T11:43:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2399999999999998e-06 2024-09-16T11:43:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515265+0000", 2024-09-16T11:43:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3909999999999998e-06 2024-09-16T11:43:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564555+0000", 2024-09-16T11:43:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049290002999999999 2024-09-16T11:43:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564628+0000", 2024-09-16T11:43:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2559999999999996e-05 2024-09-16T11:43:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564712+0000", 2024-09-16T11:43:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4040000000000002e-05 2024-09-16T11:43:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19375 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3301376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40734426499999998, 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049606631999999998, 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19375 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:43:55.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:43:55.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:43:55.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:43:55.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515271+0000", 2024-09-16T11:43:55.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5230000000000001e-06 2024-09-16T11:43:55.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515276+0000", 2024-09-16T11:43:55.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4650000000000002e-06 2024-09-16T11:43:55.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515279+0000", 2024-09-16T11:43:55.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0630000000000002e-06 2024-09-16T11:43:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564738+0000", 2024-09-16T11:43:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049459001000000002 2024-09-16T11:43:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564780+0000", 2024-09-16T11:43:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1693999999999998e-05 2024-09-16T11:43:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564876+0000", 2024-09-16T11:43:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5571999999999999e-05 2024-09-16T11:43:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.967 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19376 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3727360~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40733035299999998, 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049720590000000002, 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19376 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517103+0000", 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001820276 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517108+0000", 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5410000000000002e-06 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517120+0000", 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2092e-05 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517125+0000", 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6829999999999996e-06 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564870+0000", 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047745040000000002 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564921+0000", 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0538999999999998e-05 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565003+0000", 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2418999999999998e-05 2024-09-16T11:43:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19492 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 434176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.405401497, 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048926948999999997, 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19492 2024-09-16T11:43:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:43:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:43:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3099999999999999e-07 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517213+0000", 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6599999999999994e-07 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517216+0000", 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.706e-06 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517218+0000", 2024-09-16T11:43:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.035e-06 2024-09-16T11:43:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565959+0000", 2024-09-16T11:43:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048741380000000001 2024-09-16T11:43:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566036+0000", 2024-09-16T11:43:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7362999999999998e-05 2024-09-16T11:43:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566139+0000", 2024-09-16T11:43:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102268 2024-09-16T11:43:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19493 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 860160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.405379826, 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049086604999999998, 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19493 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:43:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517234+0000", 2024-09-16T11:43:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.37e-07 2024-09-16T11:43:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517236+0000", 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7150000000000001e-06 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517243+0000", 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9489999999999999e-06 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517246+0000", 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.636e-06 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566163+0000", 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048917284999999998 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566237+0000", 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3196999999999997e-05 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566320+0000", 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3286000000000002e-05 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19494 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1179648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40535921000000003, 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049259562999999999, 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19494 2024-09-16T11:43:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:43:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:43:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517256+0000", 2024-09-16T11:43:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.888e-06 2024-09-16T11:43:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517265+0000", 2024-09-16T11:43:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8629999999999997e-06 2024-09-16T11:43:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517269+0000", 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2039999999999999e-06 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566352+0000", 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049083107000000001 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566429+0000", 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7025000000000002e-05 2024-09-16T11:43:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566514+0000", 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4121000000000003e-05 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19495 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1286144~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40533978700000001, 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049492880000000003, 2024-09-16T11:43:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19495 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517274+0000", 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517276+0000", 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8950000000000001e-06 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517281+0000", 2024-09-16T11:43:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3879999999999999e-06 2024-09-16T11:43:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517285+0000", 2024-09-16T11:43:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5930000000000002e-06 2024-09-16T11:43:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566579+0000", 2024-09-16T11:43:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049294175000000003 2024-09-16T11:43:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566670+0000", 2024-09-16T11:43:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1096999999999999e-05 2024-09-16T11:43:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566766+0000", 2024-09-16T11:43:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6355999999999997e-05 2024-09-16T11:43:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19496 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1605632~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:43:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40532470199999998, 2024-09-16T11:43:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049632981999999999, 2024-09-16T11:43:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19496 2024-09-16T11:43:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:43:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:43:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517289+0000", 2024-09-16T11:43:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:43:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517290+0000", 2024-09-16T11:43:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.581e-06 2024-09-16T11:43:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517300+0000", 2024-09-16T11:43:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:43:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517304+0000", 2024-09-16T11:43:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:43:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566789+0000", 2024-09-16T11:43:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049485102000000003 2024-09-16T11:43:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566843+0000", 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3937000000000002e-05 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566921+0000", 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8397000000000002e-05 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19497 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 114688~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:43:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40530500699999999, 2024-09-16T11:43:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049761367000000001, 2024-09-16T11:43:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19497 2024-09-16T11:43:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519116+0000", 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018079439999999999 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519120+0000", 2024-09-16T11:43:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2760000000000002e-06 2024-09-16T11:43:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519134+0000", 2024-09-16T11:43:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3468999999999999e-05 2024-09-16T11:43:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519140+0000", 2024-09-16T11:43:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.516e-06 2024-09-16T11:43:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566943+0000", 2024-09-16T11:43:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047802561 2024-09-16T11:43:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566982+0000", 2024-09-16T11:43:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9269e-05 2024-09-16T11:43:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567070+0000", 2024-09-16T11:43:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7331999999999997e-05 2024-09-16T11:43:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19504 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 966656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:43:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40337652000000002, 2024-09-16T11:43:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049045863000000002, 2024-09-16T11:43:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19504 2024-09-16T11:43:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:43:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519239+0000", 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.962e-06 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519244+0000", 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5480000000000001e-06 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519248+0000", 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.641e-06 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568150+0000", 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048901864000000003 2024-09-16T11:43:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568208+0000", 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8399000000000001e-05 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568283+0000", 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4105999999999998e-05 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19505 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1392640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.403361215, 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049202304000000002, 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19505 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:43:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:43:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:43:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:43:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519254+0000", 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.761e-06 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519257+0000", 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.343e-06 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519259+0000", 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0559999999999999e-06 2024-09-16T11:43:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568307+0000", 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049047445000000002 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568382+0000", 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5574999999999994e-05 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568454+0000", 2024-09-16T11:43:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1799999999999997e-05 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19506 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1712128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40335117700000001, 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049341532, 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19506 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521102+0000", 2024-09-16T11:43:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018404700000000001 2024-09-16T11:43:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521106+0000", 2024-09-16T11:43:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8820000000000003e-06 2024-09-16T11:43:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521119+0000", 2024-09-16T11:43:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2546e-05 2024-09-16T11:43:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521125+0000", 2024-09-16T11:43:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0249999999999999e-06 2024-09-16T11:43:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568484+0000", 2024-09-16T11:43:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047359120999999997 2024-09-16T11:43:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568542+0000", 2024-09-16T11:43:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8360999999999999e-05 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568604+0000", 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1126999999999999e-05 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19515 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1499136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.40135889200000002, 2024-09-16T11:43:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048949431000000002, 2024-09-16T11:43:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19515 2024-09-16T11:43:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:43:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:43:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523130+0000", 2024-09-16T11:43:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00187604 2024-09-16T11:43:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523134+0000", 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523144+0000", 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2010000000000005e-06 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523149+0000", 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0039999999999999e-06 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570113+0000", 2024-09-16T11:43:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.046964517999999997 2024-09-16T11:43:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570157+0000", 2024-09-16T11:43:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4056e-05 2024-09-16T11:43:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570204+0000", 2024-09-16T11:43:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6591999999999997e-05 2024-09-16T11:43:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.017 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.017 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.017 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.017 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:56.017 INFO:teuthology.orchestra.run.smithi070.stdout:leaving image image_client.0-clone13 intact 2024-09-16T11:43:56.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:43:56.019 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:56.017+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:56.050 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-16T11:43:56.120 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:56.118+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:56.132 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:56.132 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:56.141 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout:2837 writesame 0x46209d1 thru 0x46254d9 data_size 0x13(0x4b09 bytes) 2024-09-16T11:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout:2838 write 0x32fd79e thru 0x3306e56 (0x96b9 bytes) 2024-09-16T11:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout:2839 trunc from 0x80f8f99 to 0x3fb7f0e 2024-09-16T11:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout:2840 read 0x1912c1b thru 0x191f85a (0xcc40 bytes) 2024-09-16T11:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:56.187 INFO:teuthology.orchestra.run.smithi070.stdout:2842 trunc from 0x3fb7f0e to 0x4c5256 2024-09-16T11:43:56.201 INFO:teuthology.orchestra.run.smithi070.stdout:2843 read 0x44cbd thru 0x4731e (0x2662 bytes) 2024-09-16T11:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout:2844 write 0xa8ad22a thru 0xa8ae45d (0x1234 bytes) 2024-09-16T11:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout:2845 write 0x622109a thru 0x6224548 (0x34af bytes) 2024-09-16T11:43:56.221 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:56.218+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout:2846 punch from 0x91b3d74 to 0x91c253e, (0xe7ca bytes) 2024-09-16T11:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout:2847 write 0xe379d46 thru 0xe37a244 (0x4ff bytes) 2024-09-16T11:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout:2850 write 0x7c6994 thru 0x7d6428 (0xfa95 bytes) 2024-09-16T11:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout:2851 trunc from 0xe37a245 to 0xaf69f64 2024-09-16T11:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout:2852 write 0x52ba21d thru 0x52bf083 (0x4e67 bytes) 2024-09-16T11:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout:2854 writesame 0x8f69ef3 thru 0x8f79105 data_size 0x5b(0xf213 bytes) 2024-09-16T11:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout:2856 read 0x9ee1fc5 thru 0x9ee7986 (0x59c2 bytes) 2024-09-16T11:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout:2857 punch from 0x314d466 to 0x3155170, (0x7d0a bytes) 2024-09-16T11:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout:2858 write 0x350f4b0 thru 0x3515bf2 (0x6743 bytes) 2024-09-16T11:43:56.270 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout:2862 write 0xc626b9b thru 0xc633553 (0xc9b9 bytes) 2024-09-16T11:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout:2863 write 0x3c7cca9 thru 0x3c7d0b9 (0x411 bytes) 2024-09-16T11:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout:2865 write 0x3b56689 thru 0x3b58a81 (0x23f9 bytes) 2024-09-16T11:43:56.286 INFO:teuthology.orchestra.run.smithi070.stdout:2866 read 0xa002c84 thru 0xa00f667 (0xc9e4 bytes) 2024-09-16T11:43:56.286 INFO:teuthology.orchestra.run.smithi070.stdout:2867 write 0x9304005 thru 0x930a2ac (0x62a8 bytes) 2024-09-16T11:43:56.289 INFO:teuthology.orchestra.run.smithi070.stdout:2868 read 0x1840acd thru 0x184b4d2 (0xaa06 bytes) 2024-09-16T11:43:56.292 INFO:teuthology.orchestra.run.smithi070.stdout:2869 trunc from 0xc633554 to 0xeb978cb 2024-09-16T11:43:56.297 INFO:teuthology.orchestra.run.smithi070.stdout:2870 punch from 0x6c52471 to 0x6c54ff6, (0x2b85 bytes) 2024-09-16T11:43:56.297 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:56.297 INFO:teuthology.orchestra.run.smithi070.stdout:2873 writesame 0x2b492dc thru 0x2b4f4ef data_size 0x2(0x6214 bytes) 2024-09-16T11:43:56.305 INFO:teuthology.orchestra.run.smithi070.stdout:2874 trunc from 0xeb978cb to 0xd591803 2024-09-16T11:43:56.310 INFO:teuthology.orchestra.run.smithi070.stdout:2876 punch from 0x3067e4a to 0x3072b05, (0xacbb bytes) 2024-09-16T11:43:56.322 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:56.320+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:56.325 INFO:teuthology.orchestra.run.smithi070.stdout:2877 trunc from 0xd591803 to 0xc902f3 2024-09-16T11:43:56.338 INFO:teuthology.orchestra.run.smithi070.stdout:2878 read 0x7083cf thru 0x70d4d4 (0x5106 bytes) 2024-09-16T11:43:56.338 INFO:teuthology.orchestra.run.smithi070.stdout:2879 read 0xc10255 thru 0xc18cb7 (0x8a63 bytes) 2024-09-16T11:43:56.352 INFO:teuthology.orchestra.run.smithi070.stdout:2880 writesame 0x6305e10 thru 0x6306234 data_size 0x1(0x425 bytes) 2024-09-16T11:43:56.352 INFO:teuthology.orchestra.run.smithi070.stdout:2881 punch from 0x503005a to 0x50379f6, (0x799c bytes) 2024-09-16T11:43:56.364 INFO:teuthology.orchestra.run.smithi070.stdout:2882 write 0xc6687ee thru 0xc676a5c (0xe26f bytes) 2024-09-16T11:43:56.368 INFO:teuthology.orchestra.run.smithi070.stdout:2883 clone 15 order 18 su 131072 sc 2 2024-09-16T11:43:56.383 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16853 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3375104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:43:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.0329663950000001, 2024-09-16T11:43:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027791907000000001, 2024-09-16T11:43:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16853 2024-09-16T11:43:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:43:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:43:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347568+0000", 2024-09-16T11:43:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:43:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347570+0000", 2024-09-16T11:43:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:43:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347588+0000", 2024-09-16T11:43:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7914000000000002e-05 2024-09-16T11:43:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347592+0000", 2024-09-16T11:43:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3150000000000002e-06 2024-09-16T11:43:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375199+0000", 2024-09-16T11:43:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027606065999999999 2024-09-16T11:43:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375276+0000", 2024-09-16T11:43:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7527000000000005e-05 2024-09-16T11:43:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375359+0000", 2024-09-16T11:43:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3238000000000005e-05 2024-09-16T11:43:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16854 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:43:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.0329366980000001, 2024-09-16T11:43:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027945148999999999, 2024-09-16T11:43:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16854 2024-09-16T11:43:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:43:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:43:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347598+0000", 2024-09-16T11:43:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:43:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347600+0000", 2024-09-16T11:43:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:43:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347606+0000", 2024-09-16T11:43:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0580000000000003e-06 2024-09-16T11:43:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347609+0000", 2024-09-16T11:43:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:43:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375386+0000", 2024-09-16T11:43:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027776809 2024-09-16T11:43:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375461+0000", 2024-09-16T11:43:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4868000000000002e-05 2024-09-16T11:43:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375542+0000", 2024-09-16T11:43:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1253000000000003e-05 2024-09-16T11:43:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16855 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 4120576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:43:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.032920359, 2024-09-16T11:43:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028111094999999999, 2024-09-16T11:43:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16855 2024-09-16T11:43:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:43:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:43:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347614+0000", 2024-09-16T11:43:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:43:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347616+0000", 2024-09-16T11:43:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:43:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347623+0000", 2024-09-16T11:43:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0099999999999998e-06 2024-09-16T11:43:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347626+0000", 2024-09-16T11:43:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2140000000000001e-06 2024-09-16T11:43:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375569+0000", 2024-09-16T11:43:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027942649999999999 2024-09-16T11:43:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375643+0000", 2024-09-16T11:43:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3949999999999995e-05 2024-09-16T11:43:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375725+0000", 2024-09-16T11:43:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1992999999999999e-05 2024-09-16T11:43:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16856 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3801088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:43:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.0328982879999999, 2024-09-16T11:43:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028268272000000001, 2024-09-16T11:43:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16856 2024-09-16T11:43:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:43:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:43:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347636+0000", 2024-09-16T11:43:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:43:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347638+0000", 2024-09-16T11:43:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.86e-06 2024-09-16T11:43:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347644+0000", 2024-09-16T11:43:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1739999999999997e-06 2024-09-16T11:43:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347656+0000", 2024-09-16T11:43:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2198000000000001e-05 2024-09-16T11:43:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375751+0000", 2024-09-16T11:43:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028094507000000001 2024-09-16T11:43:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375824+0000", 2024-09-16T11:43:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3178999999999998e-05 2024-09-16T11:43:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375904+0000", 2024-09-16T11:43:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9985000000000001e-05 2024-09-16T11:43:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16857 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:43:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.0328717860000001, 2024-09-16T11:43:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028275403000000001, 2024-09-16T11:43:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:43:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16857 2024-09-16T11:43:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:43:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:43:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:43:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.46e-07 2024-09-16T11:43:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347666+0000", 2024-09-16T11:43:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8739999999999997e-06 2024-09-16T11:43:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347672+0000", 2024-09-16T11:43:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1059999999999996e-06 2024-09-16T11:43:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347676+0000", 2024-09-16T11:43:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:43:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375930+0000", 2024-09-16T11:43:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028254042 2024-09-16T11:43:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375937+0000", 2024-09-16T11:43:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5519999999999999e-06 2024-09-16T11:43:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16858 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:43:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.0328541410000001, 2024-09-16T11:43:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028284592000000001, 2024-09-16T11:43:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:43:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16858 2024-09-16T11:43:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:43:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:43:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:43:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:43:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347683+0000", 2024-09-16T11:43:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6809999999999998e-06 2024-09-16T11:43:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347689+0000", 2024-09-16T11:43:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:43:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347695+0000", 2024-09-16T11:43:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0020000000000003e-06 2024-09-16T11:43:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375958+0000", 2024-09-16T11:43:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028263870999999999 2024-09-16T11:43:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375964+0000", 2024-09-16T11:43:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8209999999999998e-06 2024-09-16T11:43:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16992 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:43:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.031279453, 2024-09-16T11:43:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027704776, 2024-09-16T11:43:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:43:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16992 2024-09-16T11:43:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:43:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:43:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349255+0000", 2024-09-16T11:43:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:43:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349260+0000", 2024-09-16T11:43:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5589999999999996e-06 2024-09-16T11:43:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349267+0000", 2024-09-16T11:43:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4019999999999999e-06 2024-09-16T11:43:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349270+0000", 2024-09-16T11:43:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6780000000000002e-06 2024-09-16T11:43:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376953+0000", 2024-09-16T11:43:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682933 2024-09-16T11:43:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376959+0000", 2024-09-16T11:43:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:43:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.422 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.422 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.422 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16993 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.422 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:43:56.422 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.0312581279999999, 2024-09-16T11:43:56.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027707908999999999, 2024-09-16T11:43:56.422 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.422 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:43:56.422 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16993 2024-09-16T11:43:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:43:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:43:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:43:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:43:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349278+0000", 2024-09-16T11:43:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.629e-06 2024-09-16T11:43:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349291+0000", 2024-09-16T11:43:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614e-05 2024-09-16T11:43:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349295+0000", 2024-09-16T11:43:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1189999999999999e-06 2024-09-16T11:43:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376978+0000", 2024-09-16T11:43:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682445 2024-09-16T11:43:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376984+0000", 2024-09-16T11:43:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7590000000000003e-06 2024-09-16T11:43:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16994 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:43:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.0312341169999999, 2024-09-16T11:43:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027708753999999999, 2024-09-16T11:43:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:43:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16994 2024-09-16T11:43:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:43:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:43:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:43:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:43:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349302+0000", 2024-09-16T11:43:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.722e-06 2024-09-16T11:43:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349308+0000", 2024-09-16T11:43:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:43:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349312+0000", 2024-09-16T11:43:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5310000000000002e-06 2024-09-16T11:43:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377003+0000", 2024-09-16T11:43:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027691308000000001 2024-09-16T11:43:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377008+0000", 2024-09-16T11:43:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4550000000000003e-06 2024-09-16T11:43:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16999 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:43:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.031091239, 2024-09-16T11:43:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027705270000000001, 2024-09-16T11:43:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:43:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16999 2024-09-16T11:43:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:43:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:43:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:43:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.46e-07 2024-09-16T11:43:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349445+0000", 2024-09-16T11:43:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.637e-06 2024-09-16T11:43:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349451+0000", 2024-09-16T11:43:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9959999999999999e-06 2024-09-16T11:43:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349454+0000", 2024-09-16T11:43:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.647e-06 2024-09-16T11:43:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377142+0000", 2024-09-16T11:43:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027687942 2024-09-16T11:43:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377148+0000", 2024-09-16T11:43:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7019999999999998e-06 2024-09-16T11:43:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17000 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:43:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.0310757319999999, 2024-09-16T11:43:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.02771881, 2024-09-16T11:43:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:43:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17000 2024-09-16T11:43:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:43:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:43:56.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:43:56.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0600000000000001e-07 2024-09-16T11:43:56.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349460+0000", 2024-09-16T11:43:56.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.736e-06 2024-09-16T11:43:56.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349466+0000", 2024-09-16T11:43:56.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2419999999999997e-06 2024-09-16T11:43:56.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349470+0000", 2024-09-16T11:43:56.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5480000000000002e-06 2024-09-16T11:43:56.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377171+0000", 2024-09-16T11:43:56.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027701282000000001 2024-09-16T11:43:56.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377177+0000", 2024-09-16T11:43:56.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5960000000000003e-06 2024-09-16T11:43:56.442 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.442 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.442 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.443 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17001 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.443 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:43:56.443 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.0310599819999999, 2024-09-16T11:43:56.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027739243, 2024-09-16T11:43:56.444 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.444 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:43:56.444 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.444 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.444 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.444 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17001 2024-09-16T11:43:56.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.444 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:43:56.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:43:56.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:43:56.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:43:56.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349476+0000", 2024-09-16T11:43:56.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T11:43:56.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349482+0000", 2024-09-16T11:43:56.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2389999999999999e-06 2024-09-16T11:43:56.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349486+0000", 2024-09-16T11:43:56.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5980000000000001e-06 2024-09-16T11:43:56.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377206+0000", 2024-09-16T11:43:56.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027720037999999999 2024-09-16T11:43:56.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377213+0000", 2024-09-16T11:43:56.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3440000000000003e-06 2024-09-16T11:43:56.448 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.448 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.448 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.449 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17433 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 5898240~4888] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.449 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:43:56.449 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.99509632800000003, 2024-09-16T11:43:56.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027709398, 2024-09-16T11:43:56.449 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.449 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.449 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.449 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.449 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.449 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17433 2024-09-16T11:43:56.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.450 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:43:56.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:43:56.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385438+0000", 2024-09-16T11:43:56.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:43:56.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385439+0000", 2024-09-16T11:43:56.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5120000000000001e-06 2024-09-16T11:43:56.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385445+0000", 2024-09-16T11:43:56.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4e-06 2024-09-16T11:43:56.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385448+0000", 2024-09-16T11:43:56.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1669999999999999e-06 2024-09-16T11:43:56.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412916+0000", 2024-09-16T11:43:56.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027468137 2024-09-16T11:43:56.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412988+0000", 2024-09-16T11:43:56.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1606000000000005e-05 2024-09-16T11:43:56.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413147+0000", 2024-09-16T11:43:56.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015924299999999999 2024-09-16T11:43:56.454 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.454 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.454 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.455 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17434 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6004736~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.455 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:43:56.455 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.99508173799999999, 2024-09-16T11:43:56.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027777461, 2024-09-16T11:43:56.455 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.455 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.455 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.455 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.455 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.455 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17434 2024-09-16T11:43:56.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.456 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:43:56.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:43:56.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:43:56.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:43:56.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385454+0000", 2024-09-16T11:43:56.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6729999999999999e-06 2024-09-16T11:43:56.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385459+0000", 2024-09-16T11:43:56.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4079999999999997e-06 2024-09-16T11:43:56.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385463+0000", 2024-09-16T11:43:56.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.269e-06 2024-09-16T11:43:56.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413157+0000", 2024-09-16T11:43:56.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027694611000000001 2024-09-16T11:43:56.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413193+0000", 2024-09-16T11:43:56.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5673999999999998e-05 2024-09-16T11:43:56.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413230+0000", 2024-09-16T11:43:56.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6454e-05 2024-09-16T11:43:56.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.460 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.460 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17435 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6324224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.460 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:43:56.460 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.99506758500000003, 2024-09-16T11:43:56.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929059999999999, 2024-09-16T11:43:56.460 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.461 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.461 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17435 2024-09-16T11:43:56.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.461 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.461 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.461 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.461 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:43:56.461 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.462 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:43:56.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.462 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385467+0000", 2024-09-16T11:43:56.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:43:56.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385468+0000", 2024-09-16T11:43:56.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6300000000000001e-06 2024-09-16T11:43:56.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385474+0000", 2024-09-16T11:43:56.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1299999999999998e-06 2024-09-16T11:43:56.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385478+0000", 2024-09-16T11:43:56.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2880000000000001e-06 2024-09-16T11:43:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413257+0000", 2024-09-16T11:43:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027779608000000001 2024-09-16T11:43:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413331+0000", 2024-09-16T11:43:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3639000000000004e-05 2024-09-16T11:43:56.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413395+0000", 2024-09-16T11:43:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4424999999999994e-05 2024-09-16T11:43:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17436 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7176192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:43:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.995051923, 2024-09-16T11:43:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028016718999999999, 2024-09-16T11:43:56.466 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.466 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.466 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.466 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.466 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.466 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17436 2024-09-16T11:43:56.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.466 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.466 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:43:56.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:43:56.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:43:56.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:43:56.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385484+0000", 2024-09-16T11:43:56.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5179999999999999e-06 2024-09-16T11:43:56.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385489+0000", 2024-09-16T11:43:56.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7049999999999996e-06 2024-09-16T11:43:56.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385493+0000", 2024-09-16T11:43:56.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T11:43:56.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413422+0000", 2024-09-16T11:43:56.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929611999999999 2024-09-16T11:43:56.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413462+0000", 2024-09-16T11:43:56.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9585000000000001e-05 2024-09-16T11:43:56.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.470 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.470 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413499+0000", 2024-09-16T11:43:56.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6644999999999999e-05 2024-09-16T11:43:56.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.470 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.470 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17437 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6750208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.470 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:43:56.470 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.99503750800000001, 2024-09-16T11:43:56.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028167168999999999, 2024-09-16T11:43:56.471 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.471 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.471 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17437 2024-09-16T11:43:56.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.471 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.471 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:43:56.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:43:56.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385497+0000", 2024-09-16T11:43:56.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:43:56.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385498+0000", 2024-09-16T11:43:56.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.592e-06 2024-09-16T11:43:56.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385504+0000", 2024-09-16T11:43:56.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4090000000000002e-06 2024-09-16T11:43:56.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385513+0000", 2024-09-16T11:43:56.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4460000000000007e-06 2024-09-16T11:43:56.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413526+0000", 2024-09-16T11:43:56.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028012866000000001 2024-09-16T11:43:56.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413601+0000", 2024-09-16T11:43:56.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5388000000000004e-05 2024-09-16T11:43:56.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413663+0000", 2024-09-16T11:43:56.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.211e-05 2024-09-16T11:43:56.475 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.475 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.475 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.475 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17438 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7495680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.475 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:43:56.476 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.99501474899999998, 2024-09-16T11:43:56.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028224439, 2024-09-16T11:43:56.476 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.476 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.476 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.476 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.476 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.476 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17438 2024-09-16T11:43:56.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.476 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:43:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:43:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:43:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4900000000000001e-07 2024-09-16T11:43:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385521+0000", 2024-09-16T11:43:56.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:43:56.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385526+0000", 2024-09-16T11:43:56.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2680000000000002e-06 2024-09-16T11:43:56.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385529+0000", 2024-09-16T11:43:56.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2040000000000002e-06 2024-09-16T11:43:56.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413674+0000", 2024-09-16T11:43:56.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028144226000000001 2024-09-16T11:43:56.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413710+0000", 2024-09-16T11:43:56.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6665000000000002e-05 2024-09-16T11:43:56.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413743+0000", 2024-09-16T11:43:56.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3155999999999999e-05 2024-09-16T11:43:56.480 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.480 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.480 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.480 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17439 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7921664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.480 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:43:56.480 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.99495359500000002, 2024-09-16T11:43:56.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028326131000000001, 2024-09-16T11:43:56.481 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.481 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.481 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.481 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.481 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.481 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17439 2024-09-16T11:43:56.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.481 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:43:56.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:43:56.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385581+0000", 2024-09-16T11:43:56.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9299999999999998e-07 2024-09-16T11:43:56.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385583+0000", 2024-09-16T11:43:56.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6220000000000001e-06 2024-09-16T11:43:56.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385592+0000", 2024-09-16T11:43:56.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5220000000000007e-06 2024-09-16T11:43:56.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385595+0000", 2024-09-16T11:43:56.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8669999999999998e-06 2024-09-16T11:43:56.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413771+0000", 2024-09-16T11:43:56.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028175875999999999 2024-09-16T11:43:56.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413845+0000", 2024-09-16T11:43:56.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4382999999999996e-05 2024-09-16T11:43:56.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413906+0000", 2024-09-16T11:43:56.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1367999999999999e-05 2024-09-16T11:43:56.485 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.485 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.485 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.486 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17440 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.486 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:43:56.486 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.99493638699999998, 2024-09-16T11:43:56.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028391117, 2024-09-16T11:43:56.486 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.486 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.486 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.486 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.486 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.486 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17440 2024-09-16T11:43:56.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.487 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:43:56.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:43:56.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385598+0000", 2024-09-16T11:43:56.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:43:56.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385603+0000", 2024-09-16T11:43:56.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:43:56.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385607+0000", 2024-09-16T11:43:56.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4869999999999998e-06 2024-09-16T11:43:56.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385609+0000", 2024-09-16T11:43:56.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2759999999999999e-06 2024-09-16T11:43:56.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413916+0000", 2024-09-16T11:43:56.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028307253000000001 2024-09-16T11:43:56.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413955+0000", 2024-09-16T11:43:56.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8652000000000002e-05 2024-09-16T11:43:56.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413989+0000", 2024-09-16T11:43:56.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3587999999999998e-05 2024-09-16T11:43:56.491 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.491 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.491 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.491 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.491 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.491 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:56.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:43:56.493 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:56.491+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:56.593 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:56.591+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:56.631 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:56.631 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:56.632 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:43:56.632 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:56.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:43:56.694 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:56.692+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:56.737 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:56.737 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:43:56.737 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:43:56.737 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:43:56.738 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:56.746 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:43:56.795 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:56.793+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:56.844 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:43:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:43:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:43:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:43:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19545 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4415488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:43:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.317685164, 2024-09-16T11:43:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043429581000000002, 2024-09-16T11:43:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19545 2024-09-16T11:43:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:43:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:43:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:43:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:43:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523839+0000", 2024-09-16T11:43:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0220000000000004e-06 2024-09-16T11:43:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523845+0000", 2024-09-16T11:43:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6500000000000001e-06 2024-09-16T11:43:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523848+0000", 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1879999999999998e-06 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567092+0000", 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043244145999999997 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567176+0000", 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3825999999999993e-05 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567263+0000", 2024-09-16T11:43:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7423999999999995e-05 2024-09-16T11:43:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19546 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4734976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:43:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.3176673239999999, 2024-09-16T11:43:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043599231000000002, 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19546 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:43:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523954+0000", 2024-09-16T11:43:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010217099999999999 2024-09-16T11:43:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523957+0000", 2024-09-16T11:43:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3270000000000001e-06 2024-09-16T11:43:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523966+0000", 2024-09-16T11:43:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6489999999999997e-06 2024-09-16T11:43:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523971+0000", 2024-09-16T11:43:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:43:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567291+0000", 2024-09-16T11:43:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043319931999999998 2024-09-16T11:43:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567360+0000", 2024-09-16T11:43:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8628999999999996e-05 2024-09-16T11:43:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567451+0000", 2024-09-16T11:43:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1012000000000002e-05 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19547 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4841472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.317542467, 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043658858000000002, 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19547 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525102+0000", 2024-09-16T11:43:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011257730000000001 2024-09-16T11:43:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525106+0000", 2024-09-16T11:43:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9910000000000004e-06 2024-09-16T11:43:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525116+0000", 2024-09-16T11:43:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0231000000000001e-05 2024-09-16T11:43:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525137+0000", 2024-09-16T11:43:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0828000000000001e-05 2024-09-16T11:43:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567478+0000", 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.042340467999999999 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567554+0000", 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6074000000000004e-05 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567635+0000", 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1493000000000001e-05 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19553 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4947968~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:43:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.3163154050000001, 2024-09-16T11:43:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043450397000000002, 2024-09-16T11:43:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19553 2024-09-16T11:43:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:43:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:43:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:43:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1799999999999999e-07 2024-09-16T11:43:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525205+0000", 2024-09-16T11:43:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.062e-06 2024-09-16T11:43:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525208+0000", 2024-09-16T11:43:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8739999999999999e-06 2024-09-16T11:43:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525209+0000", 2024-09-16T11:43:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.725e-06 2024-09-16T11:43:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568520+0000", 2024-09-16T11:43:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043310226 2024-09-16T11:43:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568569+0000", 2024-09-16T11:43:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8959999999999999e-05 2024-09-16T11:43:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568654+0000", 2024-09-16T11:43:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5332000000000002e-05 2024-09-16T11:43:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.864 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19554 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5267456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.864 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:43:56.864 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.316306413, 2024-09-16T11:43:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043659587, 2024-09-16T11:43:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19554 2024-09-16T11:43:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:43:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:43:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527103+0000", 2024-09-16T11:43:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018907419999999999 2024-09-16T11:43:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527108+0000", 2024-09-16T11:43:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6040000000000003e-06 2024-09-16T11:43:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527118+0000", 2024-09-16T11:43:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:43:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527122+0000", 2024-09-16T11:43:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9559999999999996e-06 2024-09-16T11:43:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568688+0000", 2024-09-16T11:43:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041565116999999999 2024-09-16T11:43:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568768+0000", 2024-09-16T11:43:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0474000000000002e-05 2024-09-16T11:43:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568872+0000", 2024-09-16T11:43:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104028 2024-09-16T11:43:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19565 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6332416~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:43:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.3142278410000001, 2024-09-16T11:43:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043494890000000001, 2024-09-16T11:43:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19565 2024-09-16T11:43:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:43:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:43:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527292+0000", 2024-09-16T11:43:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.27e-07 2024-09-16T11:43:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527293+0000", 2024-09-16T11:43:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.917e-06 2024-09-16T11:43:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527299+0000", 2024-09-16T11:43:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6699999999999999e-06 2024-09-16T11:43:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527304+0000", 2024-09-16T11:43:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6160000000000004e-06 2024-09-16T11:43:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570683+0000", 2024-09-16T11:43:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043379171000000001 2024-09-16T11:43:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570733+0000", 2024-09-16T11:43:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0485000000000002e-05 2024-09-16T11:43:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570786+0000", 2024-09-16T11:43:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2604e-05 2024-09-16T11:43:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19566 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6758400~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:43:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.314211507, 2024-09-16T11:43:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043643183000000002, 2024-09-16T11:43:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19566 2024-09-16T11:43:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:43:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:43:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527308+0000", 2024-09-16T11:43:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:43:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527310+0000", 2024-09-16T11:43:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.68e-06 2024-09-16T11:43:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527316+0000", 2024-09-16T11:43:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9730000000000002e-06 2024-09-16T11:43:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527319+0000", 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.027e-06 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570810+0000", 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043491525000000003 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570882+0000", 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570951+0000", 2024-09-16T11:43:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8782000000000007e-05 2024-09-16T11:43:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19567 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:43:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.314196801, 2024-09-16T11:43:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043786291999999997, 2024-09-16T11:43:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19567 2024-09-16T11:43:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529108+0000", 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001785656 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529113+0000", 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1080000000000001e-06 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529122+0000", 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0550000000000005e-06 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529127+0000", 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9849999999999997e-06 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570997+0000", 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041869811 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571063+0000", 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6340999999999994e-05 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571108+0000", 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5336000000000001e-05 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19580 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 8036352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.3122214999999999, 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043451596000000002, 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19580 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:43:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:43:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531106+0000", 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001808179 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531110+0000", 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5990000000000004e-06 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531123+0000", 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2687000000000001e-05 2024-09-16T11:43:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531129+0000", 2024-09-16T11:43:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:43:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572642+0000", 2024-09-16T11:43:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041513474000000002 2024-09-16T11:43:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572691+0000", 2024-09-16T11:43:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8989000000000001e-05 2024-09-16T11:43:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572749+0000", 2024-09-16T11:43:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7806999999999997e-05 2024-09-16T11:43:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19708 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4530176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:43:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.310174935, 2024-09-16T11:43:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394863999999998, 2024-09-16T11:43:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19708 2024-09-16T11:43:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:43:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:43:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:43:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.67e-07 2024-09-16T11:43:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531346+0000", 2024-09-16T11:43:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.477e-06 2024-09-16T11:43:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531351+0000", 2024-09-16T11:43:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1050000000000003e-06 2024-09-16T11:43:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531354+0000", 2024-09-16T11:43:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9960000000000002e-06 2024-09-16T11:43:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574620+0000", 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043266365000000001 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574662+0000", 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1662000000000002e-05 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574739+0000", 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6991999999999997e-05 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19709 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4849664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.3101614589999999, 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043537291999999998, 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19709 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:43:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:43:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531358+0000", 2024-09-16T11:43:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:43:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531360+0000", 2024-09-16T11:43:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:43:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531365+0000", 2024-09-16T11:43:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.716e-06 2024-09-16T11:43:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531369+0000", 2024-09-16T11:43:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5410000000000001e-06 2024-09-16T11:43:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574763+0000", 2024-09-16T11:43:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394359 2024-09-16T11:43:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574831+0000", 2024-09-16T11:43:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7997999999999995e-05 2024-09-16T11:43:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574895+0000", 2024-09-16T11:43:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3427000000000001e-05 2024-09-16T11:43:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19710 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4956160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:43:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.31014618, 2024-09-16T11:43:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043674198999999997, 2024-09-16T11:43:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19710 2024-09-16T11:43:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:43:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:43:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533105+0000", 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001731961 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533109+0000", 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3080000000000001e-06 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533120+0000", 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0625000000000001e-05 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533125+0000", 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574923+0000", 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041797187 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574991+0000", 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8752999999999997e-05 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.575047+0000", 2024-09-16T11:43:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5596000000000001e-05 2024-09-16T11:43:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19719 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5488640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:43:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.3081744310000001, 2024-09-16T11:43:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043412013999999999, 2024-09-16T11:43:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19719 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533345+0000", 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533347+0000", 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8139999999999999e-06 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533352+0000", 2024-09-16T11:43:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4970000000000001e-06 2024-09-16T11:43:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533355+0000", 2024-09-16T11:43:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.711e-06 2024-09-16T11:43:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576629+0000", 2024-09-16T11:43:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043274295999999997 2024-09-16T11:43:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576706+0000", 2024-09-16T11:43:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6711000000000005e-05 2024-09-16T11:43:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576757+0000", 2024-09-16T11:43:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0509e-05 2024-09-16T11:43:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19720 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5808128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:43:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.308161782, 2024-09-16T11:43:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043577006000000001, 2024-09-16T11:43:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19720 2024-09-16T11:43:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:43:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.35e-07 2024-09-16T11:43:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533359+0000", 2024-09-16T11:43:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:43:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533364+0000", 2024-09-16T11:43:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4349999999999996e-06 2024-09-16T11:43:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533367+0000", 2024-09-16T11:43:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2349999999999999e-06 2024-09-16T11:43:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576786+0000", 2024-09-16T11:43:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043418885999999997 2024-09-16T11:43:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576862+0000", 2024-09-16T11:43:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6172000000000002e-05 2024-09-16T11:43:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576934+0000", 2024-09-16T11:43:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1891999999999995e-05 2024-09-16T11:43:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19721 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6234112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:43:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.308148262, 2024-09-16T11:43:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043681656999999999, 2024-09-16T11:43:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19721 2024-09-16T11:43:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:43:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-07 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533373+0000", 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.973e-06 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533377+0000", 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.403e-06 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533379+0000", 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.058e-06 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576959+0000", 2024-09-16T11:43:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043579433000000001 2024-09-16T11:43:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577015+0000", 2024-09-16T11:43:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5859000000000002e-05 2024-09-16T11:43:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577052+0000", 2024-09-16T11:43:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7620999999999998e-05 2024-09-16T11:43:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19722 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6340608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:43:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.308137535, 2024-09-16T11:43:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043854922999999997, 2024-09-16T11:43:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19722 2024-09-16T11:43:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:43:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:43:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535103+0000", 2024-09-16T11:43:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017217020000000001 2024-09-16T11:43:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535107+0000", 2024-09-16T11:43:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2819999999999998e-06 2024-09-16T11:43:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535118+0000", 2024-09-16T11:43:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0148e-05 2024-09-16T11:43:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535123+0000", 2024-09-16T11:43:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:43:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577091+0000", 2024-09-16T11:43:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041967930000000001 2024-09-16T11:43:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577163+0000", 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1853999999999993e-05 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577236+0000", 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3699e-05 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19732 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6447104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.3062776789999999, 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043399048000000003, 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19732 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535242+0000", 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535243+0000", 2024-09-16T11:43:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.793e-06 2024-09-16T11:43:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535249+0000", 2024-09-16T11:43:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.643e-06 2024-09-16T11:43:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535252+0000", 2024-09-16T11:43:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:43:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578501+0000", 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043248343000000002 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578576+0000", 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5544999999999996e-05 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578640+0000", 2024-09-16T11:43:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4018999999999998e-05 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19733 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6873088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.3062568379999999, 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043549681, 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19733 2024-09-16T11:43:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:43:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:43:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535263+0000", 2024-09-16T11:43:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:43:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535265+0000", 2024-09-16T11:43:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9369999999999998e-06 2024-09-16T11:43:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535271+0000", 2024-09-16T11:43:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9719999999999997e-06 2024-09-16T11:43:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535274+0000", 2024-09-16T11:43:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:43:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578668+0000", 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043393355000000002 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578744+0000", 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6853000000000004e-05 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578812+0000", 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.737e-05 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19734 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7299072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:43:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.306241091, 2024-09-16T11:43:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043699189999999999, 2024-09-16T11:43:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19734 2024-09-16T11:43:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:43:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:43:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537101+0000", 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001823098 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537105+0000", 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537119+0000", 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614999999999999e-05 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537125+0000", 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578838+0000", 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041712605999999999 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578912+0000", 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4396999999999999e-05 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578977+0000", 2024-09-16T11:43:56.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4858999999999997e-05 2024-09-16T11:43:56.936 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.936 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.936 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.936 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19851 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4431872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:56.936 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:43:56.936 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.3041603470000001, 2024-09-16T11:43:56.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043564540999999998, 2024-09-16T11:43:56.936 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:43:56.937 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:43:56.937 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:43:56.937 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:43:56.937 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:56.937 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19851 2024-09-16T11:43:56.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.937 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:43:56.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:43:56.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:43:56.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539100+0000", 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017412079999999999 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539105+0000", 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T11:43:56.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539116+0000", 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1824999999999999e-05 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539122+0000", 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9859999999999999e-06 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580764+0000", 2024-09-16T11:43:56.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041641299 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580839+0000", 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5167999999999996e-05 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580923+0000", 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4314999999999995e-05 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.940 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:43:56.941 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:43:56.941 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:43:56.941 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:43:56.943 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:43:56.943 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone14 from 0xc676a5d (overlap 0x4c5256) to 0x0 2024-09-16T11:43:56.944 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:56.940+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:57.038 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-16T11:43:57.043 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:57.041+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:57.048 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:57.048 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:57.048 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:43:57.048 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:57.057 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:43:57.144 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:57.142+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:57.193 INFO:teuthology.orchestra.run.smithi070.stdout:2884 trunc from 0xc676a5d to 0x99d6f5c 2024-09-16T11:43:57.195 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:57.195 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:57.195 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:57.195 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:57.195 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:57.205 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:43:57.206 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:57.216 INFO:teuthology.orchestra.run.smithi070.stdout:2886 write 0xdb44ab7 thru 0xdb488a6 (0x3df0 bytes) 2024-09-16T11:43:57.220 INFO:teuthology.orchestra.run.smithi070.stdout:2887 clone 16 order 20 su 8192 sc 6 2024-09-16T11:43:57.245 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:57.243+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:57.343 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:43:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:43:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:43:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:43:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.979723648, 2024-09-16T11:43:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:43:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:43:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:43:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:43:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:43:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:43:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:43:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:43:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.9797016190000001, 2024-09-16T11:43:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:43:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:43:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:43:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:43:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:43:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:43:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:43:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:43:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:43:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:43:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:43:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:43:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:43:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:43:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:43:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:43:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:43:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:43:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:43:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.9796868540000001, 2024-09-16T11:43:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:43:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:43:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:43:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:43:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:43:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:43:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:43:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:43:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:43:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:43:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:43:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:43:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:43:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:43:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:43:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:43:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:43:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:43:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:43:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.9796667860000001, 2024-09-16T11:43:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:43:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:43:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:43:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:43:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:43:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:43:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:43:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:43:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:43:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.97963595, 2024-09-16T11:43:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:43:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:43:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:43:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:43:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:43:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:43:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:43:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:43:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:43:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.9796164300000001, 2024-09-16T11:43:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:43:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:43:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:43:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:43:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:43:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:43:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:43:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:43:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:43:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:43:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:43:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:43:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:43:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:43:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:43:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:43:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:43:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:43:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:43:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.9795888509999999, 2024-09-16T11:43:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:43:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:43:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:43:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:43:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:43:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:43:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:43:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:43:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.9795701559999999, 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:43:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:43:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:43:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:43:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:43:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:43:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:43:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:43:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:43:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:43:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:43:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.9795551470000001, 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:43:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:43:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:43:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.979523049, 2024-09-16T11:43:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:43:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:43:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:43:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:43:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:43:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:43:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:43:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:43:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:43:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:43:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:43:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:43:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:43:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.9794994299999999, 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:43:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:43:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:43:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.9794847010000001, 2024-09-16T11:43:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:43:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.399 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:43:57.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.399 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:43:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:43:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:43:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:43:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:43:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:43:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:43:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:43:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:43:57.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:43:57.403 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:43:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.979462877, 2024-09-16T11:43:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:43:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:43:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:43:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:43:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:43:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:43:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:43:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:43:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:43:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:43:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:43:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:43:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:43:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:43:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:43:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.979448227, 2024-09-16T11:43:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:43:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:43:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:43:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:43:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:43:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:43:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:43:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:43:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:43:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:43:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:43:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:43:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:43:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:43:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:43:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:43:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:43:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:43:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.9794266540000001, 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:43:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:43:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:43:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:43:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:43:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:43:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:43:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:43:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:43:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:43:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:43:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.979407208, 2024-09-16T11:43:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:43:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:43:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:43:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:43:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:43:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:43:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:43:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:43:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:43:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:43:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:43:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:43:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:43:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:43:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:43:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:43:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:43:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:43:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:43:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.9793705109999999, 2024-09-16T11:43:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:43:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:43:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:43:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:43:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:43:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:43:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:43:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:43:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:43:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:43:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:43:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.979348565, 2024-09-16T11:43:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:43:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:43:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:43:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:43:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:43:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:43:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:43:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:43:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:43:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:43:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:43:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:43:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:43:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:43:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:43:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:43:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:43:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:43:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.9793266790000001, 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:43:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:43:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:43:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:43:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:43:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:43:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:43:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:43:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:43:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:43:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:43:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:43:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:43:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:43:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:43:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:43:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.97931196, 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:43:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:43:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:43:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:43:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:43:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:43:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:43:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:43:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:43:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:43:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:43:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:43:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:43:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:43:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:43:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:43:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:43:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:43:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:43:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:43:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:43:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:43:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:43:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:43:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:43:57.443 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:57.445 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:43:57.445 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:57.442+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:57.545 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:57.543+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:57.585 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:57.585 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:57.585 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:43:57.585 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:57.594 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:43:57.632 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:43:57.677 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:57.675+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:57.732 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:43:57.732 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:43:57.732 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:43:57.732 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:43:57.733 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:43:57.740 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:43:57.747 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:43:57.747 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:43:57.748 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:57.745+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:57.762 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:43:57.763 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:43:57.777 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:43:57.778 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:43:57.791 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:43:57.792 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:43:57.807 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:43:57.807 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:43:57.822 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:43:57.822 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:43:57.837 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:43:57.837 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:43:57.848 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:57.846+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:57.851 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:43:57.852 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:43:57.868 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T11:43:57.881 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:57.881 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:57.890 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:43:57.944 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone15 from 0xdb488a7 (overlap 0x99d6f5c) to 0x4ceb7ae 2024-09-16T11:43:57.948 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:57.946+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:58.010 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:58.011 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:58.023 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:43:58.068 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:43:58.066+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:43:58.138 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:58.139 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:58.147 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:43:58.189 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:58.186+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:58.197 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-16T11:43:58.198 INFO:teuthology.orchestra.run.smithi070.stdout:2888 read 0x502d6f4 thru 0x502f2e9 (0x1bf6 bytes) 2024-09-16T11:43:58.198 INFO:teuthology.orchestra.run.smithi070.stdout:2889 write 0x949c0a1 thru 0x94a67cc (0xa72c bytes) 2024-09-16T11:43:58.214 INFO:teuthology.orchestra.run.smithi070.stdout:2890 writesame 0x65e20bc thru 0x65f15db data_size 0x6a(0xf520 bytes) 2024-09-16T11:43:58.224 INFO:teuthology.orchestra.run.smithi070.stdout:2893 writesame 0x74d281a thru 0x74ddb98 data_size 0x35(0xb37f bytes) 2024-09-16T11:43:58.231 INFO:teuthology.orchestra.run.smithi070.stdout:2894 read 0xaa2d142 thru 0xaa3a128 (0xcfe7 bytes) 2024-09-16T11:43:58.231 INFO:teuthology.orchestra.run.smithi070.stdout:2896 punch from 0x4b90596 to 0x4b9f132, (0xeb9c bytes) 2024-09-16T11:43:58.231 INFO:teuthology.orchestra.run.smithi070.stdout:2897 writesame 0x6c2616f thru 0x6c29100 data_size 0x1(0x2f92 bytes) 2024-09-16T11:43:58.238 INFO:teuthology.orchestra.run.smithi070.stdout:2898 writesame 0x3fdd21a thru 0x3fdfaa9 data_size 0x3b(0x2890 bytes) 2024-09-16T11:43:58.242 INFO:teuthology.orchestra.run.smithi070.stdout:2900 writesame 0x26ae484 thru 0x26bb035 data_size 0x9(0xcbb2 bytes) 2024-09-16T11:43:58.250 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:58.248+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:58.251 INFO:teuthology.orchestra.run.smithi070.stdout:2901 read 0xdaae4a5 thru 0xdab694e (0x84aa bytes) 2024-09-16T11:43:58.251 INFO:teuthology.orchestra.run.smithi070.stdout:2902 write 0xa0b4b4a thru 0xa0bbd19 (0x71d0 bytes) 2024-09-16T11:43:58.257 INFO:teuthology.orchestra.run.smithi070.stdout:2903 write 0xa78a1ea thru 0xa790862 (0x6679 bytes) 2024-09-16T11:43:58.271 INFO:teuthology.orchestra.run.smithi070.stdout:2905 trunc from 0xdb488a7 to 0x572268b 2024-09-16T11:43:58.291 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:58.291 INFO:teuthology.orchestra.run.smithi070.stdout:2908 read 0x550593 thru 0x5512a9 (0xd17 bytes) 2024-09-16T11:43:58.291 INFO:teuthology.orchestra.run.smithi070.stdout:2910 read 0x4e3415c thru 0x4e415bb (0xd460 bytes) 2024-09-16T11:43:58.291 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:58.291 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:58.292 INFO:teuthology.orchestra.run.smithi070.stdout:2915 read 0x44a2cba thru 0x44a72f7 (0x463e bytes) 2024-09-16T11:43:58.292 INFO:teuthology.orchestra.run.smithi070.stdout:2916 read 0x3ce0336 thru 0x3ce74cf (0x719a bytes) 2024-09-16T11:43:58.292 INFO:teuthology.orchestra.run.smithi070.stdout:2917 read 0x250b1e6 thru 0x251029f (0x50ba bytes) 2024-09-16T11:43:58.292 INFO:teuthology.orchestra.run.smithi070.stdout:2918 punch from 0x3717b79 to 0x37254f5, (0xd97c bytes) 2024-09-16T11:43:58.292 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:58.297 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:58.297 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:58.305 INFO:teuthology.orchestra.run.smithi070.stdout:2921 write 0xd00af6d thru 0xd011d07 (0x6d9b bytes) 2024-09-16T11:43:58.306 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:43:58.311 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:58.312 INFO:teuthology.orchestra.run.smithi070.stdout:2923 read 0x1c17484 thru 0x1c25deb (0xe968 bytes) 2024-09-16T11:43:58.312 INFO:teuthology.orchestra.run.smithi070.stdout:2924 punch from 0xb2d4ca9 to 0xb2dd461, (0x87b8 bytes) 2024-09-16T11:43:58.312 INFO:teuthology.orchestra.run.smithi070.stdout:2925 read 0x42d40d3 thru 0x42e106a (0xcf98 bytes) 2024-09-16T11:43:58.312 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:58.317 INFO:teuthology.orchestra.run.smithi070.stdout:2927 trunc from 0xd011d08 to 0x8aca842 2024-09-16T11:43:58.325 INFO:teuthology.orchestra.run.smithi070.stdout:2928 write 0x433cadd thru 0x43413a0 (0x48c4 bytes) 2024-09-16T11:43:58.339 INFO:teuthology.orchestra.run.smithi070.stdout:2929 write 0xbaa0be7 thru 0xbaa59bd (0x4dd7 bytes) 2024-09-16T11:43:58.343 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:58.343 INFO:teuthology.orchestra.run.smithi070.stdout:2933 read 0x403a67c thru 0x4048632 (0xdfb7 bytes) 2024-09-16T11:43:58.343 INFO:teuthology.orchestra.run.smithi070.stdout:2935 read 0x3687533 thru 0x3689f29 (0x29f7 bytes) 2024-09-16T11:43:58.344 INFO:teuthology.orchestra.run.smithi070.stdout:2937 trunc from 0xbaa59be to 0xb7a26db 2024-09-16T11:43:58.351 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:58.348+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:58.351 INFO:teuthology.orchestra.run.smithi070.stdout:2938 write 0x2c0e44d thru 0x2c13d58 (0x590c bytes) 2024-09-16T11:43:58.369 INFO:teuthology.orchestra.run.smithi070.stdout:2941 trunc from 0xb7a26db to 0x2c4669e 2024-09-16T11:43:58.402 INFO:teuthology.orchestra.run.smithi070.stdout:2944 write 0xa6938fc thru 0xa695202 (0x1907 bytes) 2024-09-16T11:43:58.408 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:58.409 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:58.412 INFO:teuthology.orchestra.run.smithi070.stdout:2945 writesame 0xd07ce30 thru 0xd0813cc data_size 0x1(0x459d bytes) 2024-09-16T11:43:58.412 INFO:teuthology.orchestra.run.smithi070.stdout:2947 write 0x5daa833 thru 0x5daefb8 (0x4786 bytes) 2024-09-16T11:43:58.416 INFO:teuthology.orchestra.run.smithi070.stdout:2948 punch from 0x34815d1 to 0x3484c4d, (0x367c bytes) 2024-09-16T11:43:58.417 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:43:58.418 INFO:teuthology.orchestra.run.smithi070.stdout:2949 trunc from 0xd0813cd to 0xdf84625 2024-09-16T11:43:58.422 INFO:teuthology.orchestra.run.smithi070.stdout:2950 punch from 0xd8af22 to 0xd9305b, (0x8139 bytes) 2024-09-16T11:43:58.422 INFO:teuthology.orchestra.run.smithi070.stdout:2951 writesame 0xaec0343 thru 0xaeccd9e data_size 0x12(0xca5c bytes) 2024-09-16T11:43:58.431 INFO:teuthology.orchestra.run.smithi070.stdout:2952 write 0x1b628c4 thru 0x1b71607 (0xed44 bytes) 2024-09-16T11:43:58.445 INFO:teuthology.orchestra.run.smithi070.stdout:2953 read 0x946bdfd thru 0x946d522 (0x1726 bytes) 2024-09-16T11:43:58.445 INFO:teuthology.orchestra.run.smithi070.stdout:2955 punch from 0x4ac4597 to 0x4acf200, (0xac69 bytes) 2024-09-16T11:43:58.445 INFO:teuthology.orchestra.run.smithi070.stdout:2956 writesame 0x4ed7776 thru 0x4ee0400 data_size 0x43(0x8c8b bytes) 2024-09-16T11:43:58.450 INFO:teuthology.orchestra.run.smithi070.stdout:2958 writesame 0x7256248 thru 0x725eb22 data_size 0x5b(0x88db bytes) 2024-09-16T11:43:58.451 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:58.449+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:58.455 INFO:teuthology.orchestra.run.smithi070.stdout:2959 read 0xcb60546 thru 0xcb6b45d (0xaf18 bytes) 2024-09-16T11:43:58.456 INFO:teuthology.orchestra.run.smithi070.stdout:2960 writesame 0x7dc6d79 thru 0x7dd5a65 data_size 0x1(0xeced bytes) 2024-09-16T11:43:58.470 INFO:teuthology.orchestra.run.smithi070.stdout:2962 punch from 0xc7eda26 to 0xc7f9ccf, (0xc2a9 bytes) 2024-09-16T11:43:58.470 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:58.470 INFO:teuthology.orchestra.run.smithi070.stdout:2964 read 0x2bb56e2 thru 0x2bbcc21 (0x7540 bytes) 2024-09-16T11:43:58.471 INFO:teuthology.orchestra.run.smithi070.stdout:2965 read 0x6eaba4a thru 0x6eae63a (0x2bf1 bytes) 2024-09-16T11:43:58.471 INFO:teuthology.orchestra.run.smithi070.stdout:2966 writesame 0xc47fbae thru 0xc48784c data_size 0x1(0x7c9f bytes) 2024-09-16T11:43:58.479 INFO:teuthology.orchestra.run.smithi070.stdout:2967 write 0xc89e103 thru 0xc8a7074 (0x8f72 bytes) 2024-09-16T11:43:58.485 INFO:teuthology.orchestra.run.smithi070.stdout:2968 read 0x7cd07ab thru 0x7ce067b (0xfed1 bytes) 2024-09-16T11:43:58.500 INFO:teuthology.orchestra.run.smithi070.stdout:2969 trunc from 0xdf84625 to 0x149af95 2024-09-16T11:43:58.536 INFO:teuthology.orchestra.run.smithi070.stdout:2971 write 0x651ee4d thru 0x652ca51 (0xdc05 bytes) 2024-09-16T11:43:58.542 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:43:58.552 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:58.549+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:58.555 INFO:teuthology.orchestra.run.smithi070.stdout:2973 write 0xd5eef2a thru 0xd5f8573 (0x964a bytes) 2024-09-16T11:43:58.565 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:58.565 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:58.572 INFO:teuthology.orchestra.run.smithi070.stdout:2975 trunc from 0xd5f8574 to 0x3c8dcad 2024-09-16T11:43:58.574 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:43:58.597 INFO:teuthology.orchestra.run.smithi070.stdout:2976 write 0xc2126f2 thru 0xc214c7e (0x258d bytes) 2024-09-16T11:43:58.600 INFO:teuthology.orchestra.run.smithi070.stdout:2977 write 0x9ab4ca5 thru 0x9ab77e6 (0x2b42 bytes) 2024-09-16T11:43:58.603 INFO:teuthology.orchestra.run.smithi070.stdout:2978 clone 17 order 19 su 262144 sc 6 2024-09-16T11:43:58.652 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:58.651+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:58.672 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:58.672 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:58.681 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:43:58.754 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:58.752+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:58.779 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:58.779 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:58.788 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:43:58.873 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:58.871+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:58.934 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:58.934 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:58.943 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:43:58.956 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone16 from 0xc214c7f (overlap 0x149af95) to 0xe6c7b5 2024-09-16T11:43:58.957 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:58.954+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:59.057 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:59.054+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:59.067 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:59.068 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:59.077 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:43:59.078 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-16T11:43:59.158 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:43:59.156+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:43:59.194 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:59.195 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:59.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:43:59.204 INFO:teuthology.orchestra.run.smithi070.stdout:2979 punch from 0x57cc587 to 0x57d8f6b, (0xc9e4 bytes) 2024-09-16T11:43:59.207 INFO:teuthology.orchestra.run.smithi070.stdout:2980 writesame 0xae249de thru 0xae2fa06 data_size 0x1(0xb029 bytes) 2024-09-16T11:43:59.230 INFO:teuthology.orchestra.run.smithi070.stdout:2981 clone 18 order 22 su 524288 sc 2 2024-09-16T11:43:59.259 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:43:59.257+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:43:59.301 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:59.301 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:59.310 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:43:59.395 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:43:59.394+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:43:59.448 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:59.448 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:59.457 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:43:59.498 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:59.497+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:59.562 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:59.560+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:59.595 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:59.596 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:59.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:43:59.663 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:43:59.661+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:43:59.702 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:59.702 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:59.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:43:59.765 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:43:59.763+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:43:59.848 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:59.848 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:59.856 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:43:59.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:43:59.873 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:43:59.873 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:43:59.873 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:43:59.882 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:43:59.905 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:43:59.902+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:43:59.967 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:43:59.966+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:43:59.974 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:43:59.974 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:43:59.985 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:43:59.991 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone17 from 0xc214c7f (overlap 0xc214c7f) to 0x13687a6 2024-09-16T11:44:00.068 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:00.068 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:00.069 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:00.067+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:00.106 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:00.107 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:00.141 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:00.142 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:00.169 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:00.166+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:00.172 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:00.172 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:00.172 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:00.173 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:00.173 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:00.173 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:00.173 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:00.186 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:00.200 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:00.200 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:00.224 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:00.224 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:00.242 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:00.243 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:00.261 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:00.261 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:00.270 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:00.269+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:00.318 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:00.318 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:00.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:00.319 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:00.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.319 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19545 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4415488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.319 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:00.319 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7919956470000002, 2024-09-16T11:44:00.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043429581000000002, 2024-09-16T11:44:00.319 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.319 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.319 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.319 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.319 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.319 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19545 2024-09-16T11:44:00.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.320 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:00.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:00.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:00.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:00.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523839+0000", 2024-09-16T11:44:00.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0220000000000004e-06 2024-09-16T11:44:00.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523845+0000", 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6500000000000001e-06 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523848+0000", 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1879999999999998e-06 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567092+0000", 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043244145999999997 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567176+0000", 2024-09-16T11:44:00.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3825999999999993e-05 2024-09-16T11:44:00.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567263+0000", 2024-09-16T11:44:00.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7423999999999995e-05 2024-09-16T11:44:00.323 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.323 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.323 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.324 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19546 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4734976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.324 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:00.324 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7919778070000003, 2024-09-16T11:44:00.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043599231000000002, 2024-09-16T11:44:00.324 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.324 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.324 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.324 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.324 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.324 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19546 2024-09-16T11:44:00.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.324 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523954+0000", 2024-09-16T11:44:00.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010217099999999999 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523957+0000", 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3270000000000001e-06 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523966+0000", 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6489999999999997e-06 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523971+0000", 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567291+0000", 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043319931999999998 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567360+0000", 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8628999999999996e-05 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567451+0000", 2024-09-16T11:44:00.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1012000000000002e-05 2024-09-16T11:44:00.328 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.328 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.328 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.328 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19547 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4841472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.328 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:00.328 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.79185295, 2024-09-16T11:44:00.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043658858000000002, 2024-09-16T11:44:00.328 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.328 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19547 2024-09-16T11:44:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525102+0000", 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011257730000000001 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525106+0000", 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9910000000000004e-06 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525116+0000", 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0231000000000001e-05 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525137+0000", 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0828000000000001e-05 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567478+0000", 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.042340467999999999 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567554+0000", 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6074000000000004e-05 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567635+0000", 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1493000000000001e-05 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.332 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19553 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4947968~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7906258880000001, 2024-09-16T11:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043450397000000002, 2024-09-16T11:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19553 2024-09-16T11:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:00.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:00.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:00.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1799999999999999e-07 2024-09-16T11:44:00.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525205+0000", 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.062e-06 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525208+0000", 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8739999999999999e-06 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525209+0000", 2024-09-16T11:44:00.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.725e-06 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568520+0000", 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043310226 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568569+0000", 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8959999999999999e-05 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568654+0000", 2024-09-16T11:44:00.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5332000000000002e-05 2024-09-16T11:44:00.337 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.337 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.337 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.337 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19554 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5267456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.337 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:00.337 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7906168959999995, 2024-09-16T11:44:00.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043659587, 2024-09-16T11:44:00.337 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.337 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.337 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19554 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:00.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527103+0000", 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018907419999999999 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527108+0000", 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6040000000000003e-06 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527118+0000", 2024-09-16T11:44:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527122+0000", 2024-09-16T11:44:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9559999999999996e-06 2024-09-16T11:44:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568688+0000", 2024-09-16T11:44:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041565116999999999 2024-09-16T11:44:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568768+0000", 2024-09-16T11:44:00.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0474000000000002e-05 2024-09-16T11:44:00.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568872+0000", 2024-09-16T11:44:00.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104028 2024-09-16T11:44:00.341 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.341 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.341 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19565 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6332416~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7885383240000001, 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043494890000000001, 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19565 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:00.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:00.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527292+0000", 2024-09-16T11:44:00.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.27e-07 2024-09-16T11:44:00.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527293+0000", 2024-09-16T11:44:00.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.917e-06 2024-09-16T11:44:00.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527299+0000", 2024-09-16T11:44:00.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6699999999999999e-06 2024-09-16T11:44:00.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527304+0000", 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6160000000000004e-06 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570683+0000", 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043379171000000001 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570733+0000", 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0485000000000002e-05 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570786+0000", 2024-09-16T11:44:00.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2604e-05 2024-09-16T11:44:00.346 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.346 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.346 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.346 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19566 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6758400~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.346 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:00.346 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7885219899999996, 2024-09-16T11:44:00.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043643183000000002, 2024-09-16T11:44:00.346 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.346 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.347 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.347 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.347 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.347 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19566 2024-09-16T11:44:00.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.347 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:00.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:00.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527308+0000", 2024-09-16T11:44:00.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:44:00.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527310+0000", 2024-09-16T11:44:00.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.68e-06 2024-09-16T11:44:00.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527316+0000", 2024-09-16T11:44:00.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9730000000000002e-06 2024-09-16T11:44:00.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527319+0000", 2024-09-16T11:44:00.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.027e-06 2024-09-16T11:44:00.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570810+0000", 2024-09-16T11:44:00.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043491525000000003 2024-09-16T11:44:00.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570882+0000", 2024-09-16T11:44:00.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:44:00.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570951+0000", 2024-09-16T11:44:00.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8782000000000007e-05 2024-09-16T11:44:00.351 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.351 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.351 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.351 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19567 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.351 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:00.351 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7885072839999996, 2024-09-16T11:44:00.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043786291999999997, 2024-09-16T11:44:00.351 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.351 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.352 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.352 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19567 2024-09-16T11:44:00.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.352 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:00.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:00.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529108+0000", 2024-09-16T11:44:00.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001785656 2024-09-16T11:44:00.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529113+0000", 2024-09-16T11:44:00.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1080000000000001e-06 2024-09-16T11:44:00.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529122+0000", 2024-09-16T11:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0550000000000005e-06 2024-09-16T11:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529127+0000", 2024-09-16T11:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9849999999999997e-06 2024-09-16T11:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570997+0000", 2024-09-16T11:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041869811 2024-09-16T11:44:00.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571063+0000", 2024-09-16T11:44:00.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6340999999999994e-05 2024-09-16T11:44:00.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571108+0000", 2024-09-16T11:44:00.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5336000000000001e-05 2024-09-16T11:44:00.355 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.355 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.355 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.356 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19580 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 8036352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.356 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:00.356 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7865319829999997, 2024-09-16T11:44:00.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043451596000000002, 2024-09-16T11:44:00.356 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.356 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.356 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.356 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.356 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.356 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19580 2024-09-16T11:44:00.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.357 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:00.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:00.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531106+0000", 2024-09-16T11:44:00.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001808179 2024-09-16T11:44:00.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531110+0000", 2024-09-16T11:44:00.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5990000000000004e-06 2024-09-16T11:44:00.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531123+0000", 2024-09-16T11:44:00.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2687000000000001e-05 2024-09-16T11:44:00.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531129+0000", 2024-09-16T11:44:00.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:44:00.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572642+0000", 2024-09-16T11:44:00.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041513474000000002 2024-09-16T11:44:00.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572691+0000", 2024-09-16T11:44:00.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8989000000000001e-05 2024-09-16T11:44:00.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572749+0000", 2024-09-16T11:44:00.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7806999999999997e-05 2024-09-16T11:44:00.360 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.360 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.360 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.360 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19708 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4530176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.360 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:00.361 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.784485418, 2024-09-16T11:44:00.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394863999999998, 2024-09-16T11:44:00.361 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.361 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.361 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.361 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.361 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.361 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19708 2024-09-16T11:44:00.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.361 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:00.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:00.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:00.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.67e-07 2024-09-16T11:44:00.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531346+0000", 2024-09-16T11:44:00.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.477e-06 2024-09-16T11:44:00.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531351+0000", 2024-09-16T11:44:00.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1050000000000003e-06 2024-09-16T11:44:00.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531354+0000", 2024-09-16T11:44:00.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9960000000000002e-06 2024-09-16T11:44:00.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574620+0000", 2024-09-16T11:44:00.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043266365000000001 2024-09-16T11:44:00.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574662+0000", 2024-09-16T11:44:00.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1662000000000002e-05 2024-09-16T11:44:00.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574739+0000", 2024-09-16T11:44:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6991999999999997e-05 2024-09-16T11:44:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19709 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4849664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7844719419999997, 2024-09-16T11:44:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043537291999999998, 2024-09-16T11:44:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.365 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.366 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.366 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.366 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.366 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19709 2024-09-16T11:44:00.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.366 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:00.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:00.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531358+0000", 2024-09-16T11:44:00.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:00.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531360+0000", 2024-09-16T11:44:00.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:44:00.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531365+0000", 2024-09-16T11:44:00.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.716e-06 2024-09-16T11:44:00.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531369+0000", 2024-09-16T11:44:00.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5410000000000001e-06 2024-09-16T11:44:00.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574763+0000", 2024-09-16T11:44:00.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394359 2024-09-16T11:44:00.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574831+0000", 2024-09-16T11:44:00.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7997999999999995e-05 2024-09-16T11:44:00.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574895+0000", 2024-09-16T11:44:00.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3427000000000001e-05 2024-09-16T11:44:00.369 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.370 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.370 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.370 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19710 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4956160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.370 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:00.370 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7844566630000003, 2024-09-16T11:44:00.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043674198999999997, 2024-09-16T11:44:00.370 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.370 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.370 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.370 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.370 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.371 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19710 2024-09-16T11:44:00.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.371 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:00.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:00.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533105+0000", 2024-09-16T11:44:00.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001731961 2024-09-16T11:44:00.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533109+0000", 2024-09-16T11:44:00.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3080000000000001e-06 2024-09-16T11:44:00.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533120+0000", 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0625000000000001e-05 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533125+0000", 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574923+0000", 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041797187 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.374 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574991+0000", 2024-09-16T11:44:00.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8752999999999997e-05 2024-09-16T11:44:00.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.374 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.374 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.575047+0000", 2024-09-16T11:44:00.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5596000000000001e-05 2024-09-16T11:44:00.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.374 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.374 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19719 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5488640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.375 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:00.375 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7824849140000003, 2024-09-16T11:44:00.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043412013999999999, 2024-09-16T11:44:00.375 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.375 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.375 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19719 2024-09-16T11:44:00.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.376 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:00.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:00.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533345+0000", 2024-09-16T11:44:00.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:44:00.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533347+0000", 2024-09-16T11:44:00.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8139999999999999e-06 2024-09-16T11:44:00.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533352+0000", 2024-09-16T11:44:00.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4970000000000001e-06 2024-09-16T11:44:00.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533355+0000", 2024-09-16T11:44:00.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.711e-06 2024-09-16T11:44:00.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576629+0000", 2024-09-16T11:44:00.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043274295999999997 2024-09-16T11:44:00.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576706+0000", 2024-09-16T11:44:00.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6711000000000005e-05 2024-09-16T11:44:00.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576757+0000", 2024-09-16T11:44:00.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0509e-05 2024-09-16T11:44:00.379 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.379 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.379 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.379 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19720 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5808128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.380 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:00.380 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.782472265, 2024-09-16T11:44:00.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043577006000000001, 2024-09-16T11:44:00.380 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.380 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.380 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.380 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.380 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.380 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19720 2024-09-16T11:44:00.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.380 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:00.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:00.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:00.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.35e-07 2024-09-16T11:44:00.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533359+0000", 2024-09-16T11:44:00.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:44:00.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533364+0000", 2024-09-16T11:44:00.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4349999999999996e-06 2024-09-16T11:44:00.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533367+0000", 2024-09-16T11:44:00.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2349999999999999e-06 2024-09-16T11:44:00.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576786+0000", 2024-09-16T11:44:00.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043418885999999997 2024-09-16T11:44:00.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576862+0000", 2024-09-16T11:44:00.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6172000000000002e-05 2024-09-16T11:44:00.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576934+0000", 2024-09-16T11:44:00.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1891999999999995e-05 2024-09-16T11:44:00.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.384 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.384 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19721 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6234112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.384 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:00.384 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7824587449999996, 2024-09-16T11:44:00.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043681656999999999, 2024-09-16T11:44:00.385 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.385 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.385 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.385 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.385 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.385 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19721 2024-09-16T11:44:00.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.385 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:00.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-07 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533373+0000", 2024-09-16T11:44:00.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.973e-06 2024-09-16T11:44:00.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533377+0000", 2024-09-16T11:44:00.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.403e-06 2024-09-16T11:44:00.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533379+0000", 2024-09-16T11:44:00.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.058e-06 2024-09-16T11:44:00.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576959+0000", 2024-09-16T11:44:00.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043579433000000001 2024-09-16T11:44:00.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577015+0000", 2024-09-16T11:44:00.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5859000000000002e-05 2024-09-16T11:44:00.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577052+0000", 2024-09-16T11:44:00.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7620999999999998e-05 2024-09-16T11:44:00.389 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.389 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.389 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.389 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19722 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6340608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.389 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:00.389 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7824480180000002, 2024-09-16T11:44:00.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043854922999999997, 2024-09-16T11:44:00.389 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.389 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.389 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.389 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.390 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.390 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19722 2024-09-16T11:44:00.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.390 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:00.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:00.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535103+0000", 2024-09-16T11:44:00.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017217020000000001 2024-09-16T11:44:00.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535107+0000", 2024-09-16T11:44:00.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2819999999999998e-06 2024-09-16T11:44:00.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535118+0000", 2024-09-16T11:44:00.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0148e-05 2024-09-16T11:44:00.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535123+0000", 2024-09-16T11:44:00.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:44:00.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577091+0000", 2024-09-16T11:44:00.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041967930000000001 2024-09-16T11:44:00.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577163+0000", 2024-09-16T11:44:00.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1853999999999993e-05 2024-09-16T11:44:00.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577236+0000", 2024-09-16T11:44:00.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3699e-05 2024-09-16T11:44:00.393 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.393 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.393 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.394 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19732 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6447104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.394 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:00.394 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7805881619999999, 2024-09-16T11:44:00.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043399048000000003, 2024-09-16T11:44:00.394 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.394 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.394 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19732 2024-09-16T11:44:00.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.395 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:00.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:00.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535242+0000", 2024-09-16T11:44:00.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:00.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535243+0000", 2024-09-16T11:44:00.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.793e-06 2024-09-16T11:44:00.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535249+0000", 2024-09-16T11:44:00.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.643e-06 2024-09-16T11:44:00.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535252+0000", 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578501+0000", 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043248343000000002 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578576+0000", 2024-09-16T11:44:00.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5544999999999996e-05 2024-09-16T11:44:00.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578640+0000", 2024-09-16T11:44:00.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4018999999999998e-05 2024-09-16T11:44:00.398 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.398 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.398 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.398 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19733 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6873088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.398 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:00.398 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7805673210000004, 2024-09-16T11:44:00.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043549681, 2024-09-16T11:44:00.399 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.399 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.399 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19733 2024-09-16T11:44:00.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.399 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:00.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:00.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535263+0000", 2024-09-16T11:44:00.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:44:00.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535265+0000", 2024-09-16T11:44:00.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9369999999999998e-06 2024-09-16T11:44:00.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535271+0000", 2024-09-16T11:44:00.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9719999999999997e-06 2024-09-16T11:44:00.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535274+0000", 2024-09-16T11:44:00.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:44:00.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578668+0000", 2024-09-16T11:44:00.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043393355000000002 2024-09-16T11:44:00.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578744+0000", 2024-09-16T11:44:00.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6853000000000004e-05 2024-09-16T11:44:00.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578812+0000", 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.737e-05 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19734 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7299072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7805515740000004, 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043699189999999999, 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.403 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.404 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.404 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.404 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.404 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19734 2024-09-16T11:44:00.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.404 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:00.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:00.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537101+0000", 2024-09-16T11:44:00.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001823098 2024-09-16T11:44:00.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537105+0000", 2024-09-16T11:44:00.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:00.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537119+0000", 2024-09-16T11:44:00.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614999999999999e-05 2024-09-16T11:44:00.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537125+0000", 2024-09-16T11:44:00.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:44:00.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578838+0000", 2024-09-16T11:44:00.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041712605999999999 2024-09-16T11:44:00.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578912+0000", 2024-09-16T11:44:00.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4396999999999999e-05 2024-09-16T11:44:00.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578977+0000", 2024-09-16T11:44:00.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4858999999999997e-05 2024-09-16T11:44:00.407 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.407 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19851 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4431872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7784708299999998, 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043564540999999998, 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19851 2024-09-16T11:44:00.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:00.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539100+0000", 2024-09-16T11:44:00.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017412079999999999 2024-09-16T11:44:00.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:00.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539105+0000", 2024-09-16T11:44:00.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T11:44:00.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:00.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539116+0000", 2024-09-16T11:44:00.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1824999999999999e-05 2024-09-16T11:44:00.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539122+0000", 2024-09-16T11:44:00.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9859999999999999e-06 2024-09-16T11:44:00.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:00.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580764+0000", 2024-09-16T11:44:00.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041641299 2024-09-16T11:44:00.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:00.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580839+0000", 2024-09-16T11:44:00.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5167999999999996e-05 2024-09-16T11:44:00.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:00.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:00.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:00.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580923+0000", 2024-09-16T11:44:00.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4314999999999995e-05 2024-09-16T11:44:00.412 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.412 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.412 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.412 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:00.413 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:00.413 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:00.414 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:00.415 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:44:00.417 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:00.417 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-16T11:43:59.872724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":497496072,"num_objects":358,"num_object_clones":77,"num_object_copies":716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":358,"num_whiteouts":90,"num_read":35726,"num_read_kb":110491,"num_write":6267,"num_write_kb":82005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5116,"ondisk_log_size":5116,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2453840,"kb_used_data":12552,"kb_used_omap":13,"kb_used_meta":2441138,"kb_avail":752520880,"statfs":{"total":773094113280,"available":770581381120,"internally_reserved":0,"allocated":12853248,"data_stored":48506601,"data_compressed":3031362,"data_compressed_allocated":5767168,"data_compressed_original":45055660,"omap_allocated":13632,"internal_metadata":2499726016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34191910,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":23,"num_read":7557,"num_read_kb":29672,"num_write":493,"num_write_kb":13855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019336"},"pg_stats":[{"pgid":"4.1f","version":"47'114","reported_seq":1211,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.688487+0000","last_change":"2024-09-16T11:43:33.782279+0000","last_active":"2024-09-16T11:43:55.688487+0000","last_peered":"2024-09-16T11:43:55.688487+0000","last_clean":"2024-09-16T11:43:55.688487+0000","last_became_active":"2024-09-16T11:43:31.779428+0000","last_became_peered":"2024-09-16T11:43:31.779428+0000","last_unstale":"2024-09-16T11:43:55.688487+0000","last_undegraded":"2024-09-16T11:43:55.688487+0000","last_fullsized":"2024-09-16T11:43:55.688487+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:30.437698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090324899999999996,"stat_sum":{"num_bytes":11496953,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":1109,"num_read_kb":2504,"num_write":123,"num_write_kb":3005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"47'58","reported_seq":189,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.399067+0000","last_change":"2024-09-16T11:43:34.248024+0000","last_active":"2024-09-16T11:43:55.399067+0000","last_peered":"2024-09-16T11:43:55.399067+0000","last_clean":"2024-09-16T11:43:55.399067+0000","last_became_active":"2024-09-16T11:43:31.889706+0000","last_became_peered":"2024-09-16T11:43:31.889706+0000","last_unstale":"2024-09-16T11:43:55.399067+0000","last_undegraded":"2024-09-16T11:43:55.399067+0000","last_fullsized":"2024-09-16T11:43:55.399067+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:25.242116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027516299999999998,"stat_sum":{"num_bytes":12994322,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":144,"num_read_kb":774,"num_write":61,"num_write_kb":2788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"49'81","reported_seq":468,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T11:43:57.067433+0000","last_change":"2024-09-16T11:43:33.777888+0000","last_active":"2024-09-16T11:43:57.067433+0000","last_peered":"2024-09-16T11:43:57.067433+0000","last_clean":"2024-09-16T11:43:57.067433+0000","last_became_active":"2024-09-16T11:43:31.775021+0000","last_became_peered":"2024-09-16T11:43:31.775021+0000","last_unstale":"2024-09-16T11:43:57.067433+0000","last_undegraded":"2024-09-16T11:43:57.067433+0000","last_fullsized":"2024-09-16T11:43:57.067433+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:50.045796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075167799999999998,"stat_sum":{"num_bytes":10260120,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":378,"num_read_kb":4674,"num_write":85,"num_write_kb":5076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"47'104","reported_seq":936,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.686859+0000","last_change":"2024-09-16T11:43:33.772949+0000","last_active":"2024-09-16T11:43:55.686859+0000","last_peered":"2024-09-16T11:43:55.686859+0000","last_clean":"2024-09-16T11:43:55.686859+0000","last_became_active":"2024-09-16T11:43:31.773661+0000","last_became_peered":"2024-09-16T11:43:31.773661+0000","last_unstale":"2024-09-16T11:43:55.686859+0000","last_undegraded":"2024-09-16T11:43:55.686859+0000","last_fullsized":"2024-09-16T11:43:55.686859+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:36.062796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023334699999999999,"stat_sum":{"num_bytes":14290326,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":959,"num_read_kb":3852,"num_write":115,"num_write_kb":577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"47'176","reported_seq":1512,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.688071+0000","last_change":"2024-09-16T11:43:33.773384+0000","last_active":"2024-09-16T11:43:55.688071+0000","last_peered":"2024-09-16T11:43:55.688071+0000","last_clean":"2024-09-16T11:43:55.688071+0000","last_became_active":"2024-09-16T11:43:31.774523+0000","last_became_peered":"2024-09-16T11:43:31.774523+0000","last_unstale":"2024-09-16T11:43:55.688071+0000","last_undegraded":"2024-09-16T11:43:55.688071+0000","last_fullsized":"2024-09-16T11:43:55.688071+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:15.703792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038485,"stat_sum":{"num_bytes":22405120,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":2301,"num_read_kb":11321,"num_write":165,"num_write_kb":2626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"47'295","reported_seq":1198,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.678120+0000","last_change":"2024-09-16T11:43:33.774141+0000","last_active":"2024-09-16T11:43:55.678120+0000","last_peered":"2024-09-16T11:43:55.678120+0000","last_clean":"2024-09-16T11:43:55.678120+0000","last_became_active":"2024-09-16T11:43:31.774794+0000","last_became_peered":"2024-09-16T11:43:31.774794+0000","last_unstale":"2024-09-16T11:43:55.678120+0000","last_undegraded":"2024-09-16T11:43:55.678120+0000","last_fullsized":"2024-09-16T11:43:55.678120+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:06:57.981702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010218040000000001,"stat_sum":{"num_bytes":14278105,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":2079,"num_read_kb":2825,"num_write":458,"num_write_kb":3198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"49'455","reported_seq":545,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T11:43:56.949829+0000","last_change":"2024-09-16T11:43:33.777212+0000","last_active":"2024-09-16T11:43:56.949829+0000","last_peered":"2024-09-16T11:43:56.949829+0000","last_clean":"2024-09-16T11:43:56.949829+0000","last_became_active":"2024-09-16T11:43:31.769719+0000","last_became_peered":"2024-09-16T11:43:31.769719+0000","last_unstale":"2024-09-16T11:43:56.949829+0000","last_undegraded":"2024-09-16T11:43:56.949829+0000","last_fullsized":"2024-09-16T11:43:56.949829+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:06.911046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035706099999999999,"stat_sum":{"num_bytes":16591226,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":1859,"num_read_kb":1586,"num_write":695,"num_write_kb":1325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"47'41","reported_seq":131,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.674883+0000","last_change":"2024-09-16T11:43:33.774063+0000","last_active":"2024-09-16T11:43:55.674883+0000","last_peered":"2024-09-16T11:43:55.674883+0000","last_clean":"2024-09-16T11:43:55.674883+0000","last_became_active":"2024-09-16T11:43:31.776951+0000","last_became_peered":"2024-09-16T11:43:31.776951+0000","last_unstale":"2024-09-16T11:43:55.674883+0000","last_undegraded":"2024-09-16T11:43:55.674883+0000","last_fullsized":"2024-09-16T11:43:55.674883+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:28:39.026621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094366700000000003,"stat_sum":{"num_bytes":13519691,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":73,"num_read_kb":944,"num_write":43,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"49'140","reported_seq":788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T11:43:57.034920+0000","last_change":"2024-09-16T11:43:33.777913+0000","last_active":"2024-09-16T11:43:57.034920+0000","last_peered":"2024-09-16T11:43:57.034920+0000","last_clean":"2024-09-16T11:43:57.034920+0000","last_became_active":"2024-09-16T11:43:31.889807+0000","last_became_peered":"2024-09-16T11:43:31.889807+0000","last_unstale":"2024-09-16T11:43:57.034920+0000","last_undegraded":"2024-09-16T11:43:57.034920+0000","last_fullsized":"2024-09-16T11:43:57.034920+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:30.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071799400000000001,"stat_sum":{"num_bytes":14220336,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":1764,"num_read_kb":1988,"num_write":130,"num_write_kb":542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"47'32","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.677031+0000","last_change":"2024-09-16T11:43:34.248043+0000","last_active":"2024-09-16T11:43:55.677031+0000","last_peered":"2024-09-16T11:43:55.677031+0000","last_clean":"2024-09-16T11:43:55.677031+0000","last_became_active":"2024-09-16T11:43:31.776111+0000","last_became_peered":"2024-09-16T11:43:31.776111+0000","last_unstale":"2024-09-16T11:43:55.677031+0000","last_undegraded":"2024-09-16T11:43:55.677031+0000","last_fullsized":"2024-09-16T11:43:55.677031+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:21:28.658052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024883199999999999,"stat_sum":{"num_bytes":13406229,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":43,"num_read_kb":320,"num_write":36,"num_write_kb":2722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"49'73","reported_seq":518,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T11:43:57.067947+0000","last_change":"2024-09-16T11:43:33.785207+0000","last_active":"2024-09-16T11:43:57.067947+0000","last_peered":"2024-09-16T11:43:57.067947+0000","last_clean":"2024-09-16T11:43:57.067947+0000","last_became_active":"2024-09-16T11:43:31.889629+0000","last_became_peered":"2024-09-16T11:43:31.889629+0000","last_unstale":"2024-09-16T11:43:57.067947+0000","last_undegraded":"2024-09-16T11:43:57.067947+0000","last_fullsized":"2024-09-16T11:43:57.067947+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:53.124562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022324199999999999,"stat_sum":{"num_bytes":18322266,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":454,"num_read_kb":6293,"num_write":75,"num_write_kb":3709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"49'84","reported_seq":202,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T11:43:56.862772+0000","last_change":"2024-09-16T11:43:33.778129+0000","last_active":"2024-09-16T11:43:56.862772+0000","last_peered":"2024-09-16T11:43:56.862772+0000","last_clean":"2024-09-16T11:43:56.862772+0000","last_became_active":"2024-09-16T11:43:31.773947+0000","last_became_peered":"2024-09-16T11:43:31.773947+0000","last_unstale":"2024-09-16T11:43:56.862772+0000","last_undegraded":"2024-09-16T11:43:56.862772+0000","last_fullsized":"2024-09-16T11:43:56.862772+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:36:48.418141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085420199999999998,"stat_sum":{"num_bytes":17129005,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":102,"num_read_kb":1582,"num_write":83,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"47'237","reported_seq":683,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:54.920082+0000","last_change":"2024-09-16T11:43:33.773601+0000","last_active":"2024-09-16T11:43:54.920082+0000","last_peered":"2024-09-16T11:43:54.920082+0000","last_clean":"2024-09-16T11:43:54.920082+0000","last_became_active":"2024-09-16T11:43:31.776781+0000","last_became_peered":"2024-09-16T11:43:31.776781+0000","last_unstale":"2024-09-16T11:43:54.920082+0000","last_undegraded":"2024-09-16T11:43:54.920082+0000","last_fullsized":"2024-09-16T11:43:54.920082+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:14.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056674299999999998,"stat_sum":{"num_bytes":14467133,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":1756,"num_read_kb":3544,"num_write":318,"num_write_kb":4622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"49'67","reported_seq":1119,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T11:43:56.951566+0000","last_change":"2024-09-16T11:43:33.773531+0000","last_active":"2024-09-16T11:43:56.951566+0000","last_peered":"2024-09-16T11:43:56.951566+0000","last_clean":"2024-09-16T11:43:56.951566+0000","last_became_active":"2024-09-16T11:43:31.780164+0000","last_became_peered":"2024-09-16T11:43:31.780164+0000","last_unstale":"2024-09-16T11:43:56.951566+0000","last_undegraded":"2024-09-16T11:43:56.951566+0000","last_fullsized":"2024-09-16T11:43:56.951566+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:56.846891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033807800000000001,"stat_sum":{"num_bytes":7260000,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":1049,"num_read_kb":4945,"num_write":66,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"49'142","reported_seq":417,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T11:43:56.951929+0000","last_change":"2024-09-16T11:43:33.777531+0000","last_active":"2024-09-16T11:43:56.951929+0000","last_peered":"2024-09-16T11:43:56.951929+0000","last_clean":"2024-09-16T11:43:56.951929+0000","last_became_active":"2024-09-16T11:43:31.889754+0000","last_became_peered":"2024-09-16T11:43:31.889754+0000","last_unstale":"2024-09-16T11:43:56.951929+0000","last_undegraded":"2024-09-16T11:43:56.951929+0000","last_fullsized":"2024-09-16T11:43:56.951929+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:54.752770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062624299999999996,"stat_sum":{"num_bytes":7205837,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":732,"num_read_kb":4184,"num_write":207,"num_write_kb":870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"49'78","reported_seq":474,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T11:43:57.037307+0000","last_change":"2024-09-16T11:43:33.777407+0000","last_active":"2024-09-16T11:43:57.037307+0000","last_peered":"2024-09-16T11:43:57.037307+0000","last_clean":"2024-09-16T11:43:57.037307+0000","last_became_active":"2024-09-16T11:43:31.775962+0000","last_became_peered":"2024-09-16T11:43:31.775962+0000","last_unstale":"2024-09-16T11:43:57.037307+0000","last_undegraded":"2024-09-16T11:43:57.037307+0000","last_fullsized":"2024-09-16T11:43:57.037307+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:46.492489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062432399999999999,"stat_sum":{"num_bytes":1405830,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":907,"num_read_kb":1984,"num_write":70,"num_write_kb":1016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T11:43:53.853866+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:43:53.853866+0000","last_peered":"2024-09-16T11:43:53.853866+0000","last_clean":"2024-09-16T11:43:53.853866+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:43:53.853866+0000","last_undegraded":"2024-09-16T11:43:53.853866+0000","last_fullsized":"2024-09-16T11:43:53.853866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"47'69","reported_seq":172,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.611222+0000","last_change":"2024-09-16T11:43:33.773359+0000","last_active":"2024-09-16T11:43:55.611222+0000","last_peered":"2024-09-16T11:43:55.611222+0000","last_clean":"2024-09-16T11:43:55.611222+0000","last_became_active":"2024-09-16T11:43:31.777591+0000","last_became_peered":"2024-09-16T11:43:31.777591+0000","last_unstale":"2024-09-16T11:43:55.611222+0000","last_undegraded":"2024-09-16T11:43:55.611222+0000","last_fullsized":"2024-09-16T11:43:55.611222+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:00:39.586913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037893300000000003,"stat_sum":{"num_bytes":10567728,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":88,"num_read_kb":1536,"num_write":68,"num_write_kb":1378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.857415+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:43:55.857415+0000","last_peered":"2024-09-16T11:43:55.857415+0000","last_clean":"2024-09-16T11:43:55.857415+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:43:55.857415+0000","last_undegraded":"2024-09-16T11:43:55.857415+0000","last_fullsized":"2024-09-16T11:43:55.857415+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"48'112","reported_seq":1580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T11:43:57.029755+0000","last_change":"2024-09-16T11:43:33.777644+0000","last_active":"2024-09-16T11:43:57.029755+0000","last_peered":"2024-09-16T11:43:57.029755+0000","last_clean":"2024-09-16T11:43:57.029755+0000","last_became_active":"2024-09-16T11:43:31.892159+0000","last_became_peered":"2024-09-16T11:43:31.892159+0000","last_unstale":"2024-09-16T11:43:57.029755+0000","last_undegraded":"2024-09-16T11:43:57.029755+0000","last_fullsized":"2024-09-16T11:43:57.029755+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:42:14.042476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062271400000000001,"stat_sum":{"num_bytes":15249590,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":1897,"num_read_kb":7330,"num_write":111,"num_write_kb":3703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.859094+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:43:55.859094+0000","last_peered":"2024-09-16T11:43:55.859094+0000","last_clean":"2024-09-16T11:43:55.859094+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:43:55.859094+0000","last_undegraded":"2024-09-16T11:43:55.859094+0000","last_fullsized":"2024-09-16T11:43:55.859094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"47'111","reported_seq":251,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.675569+0000","last_change":"2024-09-16T11:43:34.247771+0000","last_active":"2024-09-16T11:43:55.675569+0000","last_peered":"2024-09-16T11:43:55.675569+0000","last_clean":"2024-09-16T11:43:55.675569+0000","last_became_active":"2024-09-16T11:43:31.777896+0000","last_became_peered":"2024-09-16T11:43:31.777896+0000","last_unstale":"2024-09-16T11:43:55.675569+0000","last_undegraded":"2024-09-16T11:43:55.675569+0000","last_fullsized":"2024-09-16T11:43:55.675569+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:35.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026844500000000002,"stat_sum":{"num_bytes":18766096,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":133,"num_read_kb":2416,"num_write":111,"num_write_kb":2138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.857384+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:43:55.857384+0000","last_peered":"2024-09-16T11:43:55.857384+0000","last_clean":"2024-09-16T11:43:55.857384+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:43:55.857384+0000","last_undegraded":"2024-09-16T11:43:55.857384+0000","last_fullsized":"2024-09-16T11:43:55.857384+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"48'55","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.997576+0000","last_change":"2024-09-16T11:43:33.774367+0000","last_active":"2024-09-16T11:43:55.997576+0000","last_peered":"2024-09-16T11:43:55.997576+0000","last_clean":"2024-09-16T11:43:55.997576+0000","last_became_active":"2024-09-16T11:43:31.778506+0000","last_became_peered":"2024-09-16T11:43:31.778506+0000","last_unstale":"2024-09-16T11:43:55.997576+0000","last_undegraded":"2024-09-16T11:43:55.997576+0000","last_fullsized":"2024-09-16T11:43:55.997576+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:12.232493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010404660000000001,"stat_sum":{"num_bytes":409672,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":39,"num_read_kb":53,"num_write":52,"num_write_kb":541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T11:43:53.850360+0000","last_change":"2024-09-16T11:43:22.438212+0000","last_active":"2024-09-16T11:43:53.850360+0000","last_peered":"2024-09-16T11:43:53.850360+0000","last_clean":"2024-09-16T11:43:53.850360+0000","last_became_active":"2024-09-16T11:43:22.437868+0000","last_became_peered":"2024-09-16T11:43:22.437868+0000","last_unstale":"2024-09-16T11:43:53.850360+0000","last_undegraded":"2024-09-16T11:43:53.850360+0000","last_fullsized":"2024-09-16T11:43:53.850360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:35.498933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":45,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.863495+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:43:55.863495+0000","last_peered":"2024-09-16T11:43:55.863495+0000","last_clean":"2024-09-16T11:43:55.863495+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:43:55.863495+0000","last_undegraded":"2024-09-16T11:43:55.863495+0000","last_fullsized":"2024-09-16T11:43:55.863495+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"47'149","reported_seq":706,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.857394+0000","last_change":"2024-09-16T11:43:33.774349+0000","last_active":"2024-09-16T11:43:55.857394+0000","last_peered":"2024-09-16T11:43:55.857394+0000","last_clean":"2024-09-16T11:43:55.857394+0000","last_became_active":"2024-09-16T11:43:31.777796+0000","last_became_peered":"2024-09-16T11:43:31.777796+0000","last_unstale":"2024-09-16T11:43:55.857394+0000","last_undegraded":"2024-09-16T11:43:55.857394+0000","last_fullsized":"2024-09-16T11:43:55.857394+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:33.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033368800000000001,"stat_sum":{"num_bytes":8146719,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1461,"num_read_kb":1526,"num_write":156,"num_write_kb":2794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.857088+0000","last_change":"2024-09-16T11:43:22.440136+0000","last_active":"2024-09-16T11:43:55.857088+0000","last_peered":"2024-09-16T11:43:55.857088+0000","last_clean":"2024-09-16T11:43:55.857088+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:43:55.857088+0000","last_undegraded":"2024-09-16T11:43:55.857088+0000","last_fullsized":"2024-09-16T11:43:55.857088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:18.461948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T11:43:53.847750+0000","last_change":"2024-09-16T11:43:16.732834+0000","last_active":"2024-09-16T11:43:53.847750+0000","last_peered":"2024-09-16T11:43:53.847750+0000","last_clean":"2024-09-16T11:43:53.847750+0000","last_became_active":"2024-09-16T11:43:14.724728+0000","last_became_peered":"2024-09-16T11:43:14.724728+0000","last_unstale":"2024-09-16T11:43:53.847750+0000","last_undegraded":"2024-09-16T11:43:53.847750+0000","last_fullsized":"2024-09-16T11:43:53.847750+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:47.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"48'267","reported_seq":2079,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T11:43:57.036293+0000","last_change":"2024-09-16T11:43:33.774126+0000","last_active":"2024-09-16T11:43:57.036293+0000","last_peered":"2024-09-16T11:43:57.036293+0000","last_clean":"2024-09-16T11:43:57.036293+0000","last_became_active":"2024-09-16T11:43:31.777967+0000","last_became_peered":"2024-09-16T11:43:31.777967+0000","last_unstale":"2024-09-16T11:43:57.036293+0000","last_undegraded":"2024-09-16T11:43:57.036293+0000","last_fullsized":"2024-09-16T11:43:57.036293+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:43.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000373488,"stat_sum":{"num_bytes":45942003,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":2575,"num_read_kb":12286,"num_write":386,"num_write_kb":3951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T11:43:53.881687+0000","last_change":"2024-09-16T11:43:22.437965+0000","last_active":"2024-09-16T11:43:53.881687+0000","last_peered":"2024-09-16T11:43:53.881687+0000","last_clean":"2024-09-16T11:43:53.881687+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:43:53.881687+0000","last_undegraded":"2024-09-16T11:43:53.881687+0000","last_fullsized":"2024-09-16T11:43:53.881687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:38:39.799255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.859104+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:43:55.859104+0000","last_peered":"2024-09-16T11:43:55.859104+0000","last_clean":"2024-09-16T11:43:55.859104+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:43:55.859104+0000","last_undegraded":"2024-09-16T11:43:55.859104+0000","last_fullsized":"2024-09-16T11:43:55.859104+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"47'72","reported_seq":285,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.644318+0000","last_change":"2024-09-16T11:43:34.248254+0000","last_active":"2024-09-16T11:43:55.644318+0000","last_peered":"2024-09-16T11:43:55.644318+0000","last_clean":"2024-09-16T11:43:55.644318+0000","last_became_active":"2024-09-16T11:43:31.776645+0000","last_became_peered":"2024-09-16T11:43:31.776645+0000","last_unstale":"2024-09-16T11:43:55.644318+0000","last_undegraded":"2024-09-16T11:43:55.644318+0000","last_fullsized":"2024-09-16T11:43:55.644318+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:53.198323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038670399999999998,"stat_sum":{"num_bytes":34258031,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":213,"num_read_kb":3060,"num_write":66,"num_write_kb":2236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.854891+0000","last_change":"2024-09-16T11:43:22.587076+0000","last_active":"2024-09-16T11:43:55.854891+0000","last_peered":"2024-09-16T11:43:55.854891+0000","last_clean":"2024-09-16T11:43:55.854891+0000","last_became_active":"2024-09-16T11:43:22.586861+0000","last_became_peered":"2024-09-16T11:43:22.586861+0000","last_unstale":"2024-09-16T11:43:55.854891+0000","last_undegraded":"2024-09-16T11:43:55.854891+0000","last_fullsized":"2024-09-16T11:43:55.854891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:54:46.029558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":44,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.863458+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:43:55.863458+0000","last_peered":"2024-09-16T11:43:55.863458+0000","last_clean":"2024-09-16T11:43:55.863458+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:43:55.863458+0000","last_undegraded":"2024-09-16T11:43:55.863458+0000","last_fullsized":"2024-09-16T11:43:55.863458+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":130,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.859128+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:43:55.859128+0000","last_peered":"2024-09-16T11:43:55.859128+0000","last_clean":"2024-09-16T11:43:55.859128+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:43:55.859128+0000","last_undegraded":"2024-09-16T11:43:55.859128+0000","last_fullsized":"2024-09-16T11:43:55.859128+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"46'265","reported_seq":1029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.591306+0000","last_change":"2024-09-16T11:43:33.781611+0000","last_active":"2024-09-16T11:43:55.591306+0000","last_peered":"2024-09-16T11:43:55.591306+0000","last_clean":"2024-09-16T11:43:55.591306+0000","last_became_active":"2024-09-16T11:43:31.782071+0000","last_became_peered":"2024-09-16T11:43:31.782071+0000","last_unstale":"2024-09-16T11:43:55.591306+0000","last_undegraded":"2024-09-16T11:43:55.591306+0000","last_fullsized":"2024-09-16T11:43:55.591306+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:15.516215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033637000000000001,"stat_sum":{"num_bytes":51204234,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":2,"num_read":1379,"num_read_kb":7392,"num_write":338,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"49'341","reported_seq":1660,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T11:43:56.949351+0000","last_change":"2024-09-16T11:43:33.785095+0000","last_active":"2024-09-16T11:43:56.949351+0000","last_peered":"2024-09-16T11:43:56.949351+0000","last_clean":"2024-09-16T11:43:56.949351+0000","last_became_active":"2024-09-16T11:43:31.892119+0000","last_became_peered":"2024-09-16T11:43:31.892119+0000","last_unstale":"2024-09-16T11:43:56.949351+0000","last_undegraded":"2024-09-16T11:43:56.949351+0000","last_fullsized":"2024-09-16T11:43:56.949351+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:04.609697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398253,"stat_sum":{"num_bytes":20801466,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":2661,"num_read_kb":9156,"num_write":426,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"47'331","reported_seq":1909,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.644246+0000","last_change":"2024-09-16T11:43:33.773950+0000","last_active":"2024-09-16T11:43:55.644246+0000","last_peered":"2024-09-16T11:43:55.644246+0000","last_clean":"2024-09-16T11:43:55.644246+0000","last_became_active":"2024-09-16T11:43:31.775309+0000","last_became_peered":"2024-09-16T11:43:31.775309+0000","last_unstale":"2024-09-16T11:43:55.644246+0000","last_undegraded":"2024-09-16T11:43:55.644246+0000","last_fullsized":"2024-09-16T11:43:55.644246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:23.106465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086187000000000004,"stat_sum":{"num_bytes":28645364,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":4333,"num_read_kb":4861,"num_write":330,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"47'87","reported_seq":222,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.112606+0000","last_change":"2024-09-16T11:43:33.781852+0000","last_active":"2024-09-16T11:43:55.112606+0000","last_peered":"2024-09-16T11:43:55.112606+0000","last_clean":"2024-09-16T11:43:55.112606+0000","last_became_active":"2024-09-16T11:43:31.779384+0000","last_became_peered":"2024-09-16T11:43:31.779384+0000","last_unstale":"2024-09-16T11:43:55.112606+0000","last_undegraded":"2024-09-16T11:43:55.112606+0000","last_fullsized":"2024-09-16T11:43:55.112606+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:34.933146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029212900000000002,"stat_sum":{"num_bytes":9175786,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":136,"num_read_kb":460,"num_write":83,"num_write_kb":690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"47'112","reported_seq":345,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.677396+0000","last_change":"2024-09-16T11:43:34.247758+0000","last_active":"2024-09-16T11:43:55.677396+0000","last_peered":"2024-09-16T11:43:55.677396+0000","last_clean":"2024-09-16T11:43:55.677396+0000","last_became_active":"2024-09-16T11:43:31.775335+0000","last_became_peered":"2024-09-16T11:43:31.775335+0000","last_unstale":"2024-09-16T11:43:55.677396+0000","last_undegraded":"2024-09-16T11:43:55.677396+0000","last_fullsized":"2024-09-16T11:43:55.677396+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:07.521713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026563699999999999,"stat_sum":{"num_bytes":10652260,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":476,"num_read_kb":833,"num_write":122,"num_write_kb":5476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"47'176","reported_seq":593,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:54.924599+0000","last_change":"2024-09-16T11:43:33.773570+0000","last_active":"2024-09-16T11:43:54.924599+0000","last_peered":"2024-09-16T11:43:54.924599+0000","last_clean":"2024-09-16T11:43:54.924599+0000","last_became_active":"2024-09-16T11:43:31.774601+0000","last_became_peered":"2024-09-16T11:43:31.774601+0000","last_unstale":"2024-09-16T11:43:54.924599+0000","last_undegraded":"2024-09-16T11:43:54.924599+0000","last_fullsized":"2024-09-16T11:43:54.924599+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:12.584420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055828300000000002,"stat_sum":{"num_bytes":8804342,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":1092,"num_read_kb":1319,"num_write":184,"num_write_kb":1113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"47'415","reported_seq":580,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.436282+0000","last_change":"2024-09-16T11:43:33.773852+0000","last_active":"2024-09-16T11:43:55.436282+0000","last_peered":"2024-09-16T11:43:55.436282+0000","last_clean":"2024-09-16T11:43:55.436282+0000","last_became_active":"2024-09-16T11:43:31.772506+0000","last_became_peered":"2024-09-16T11:43:31.772506+0000","last_unstale":"2024-09-16T11:43:55.436282+0000","last_undegraded":"2024-09-16T11:43:55.436282+0000","last_fullsized":"2024-09-16T11:43:55.436282+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:36.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076283899999999997,"stat_sum":{"num_bytes":14075941,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":1805,"num_read_kb":2291,"num_write":648,"num_write_kb":4481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"47'228","reported_seq":573,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T11:43:55.687922+0000","last_change":"2024-09-16T11:43:33.773770+0000","last_active":"2024-09-16T11:43:55.687922+0000","last_peered":"2024-09-16T11:43:55.687922+0000","last_clean":"2024-09-16T11:43:55.687922+0000","last_became_active":"2024-09-16T11:43:31.779613+0000","last_became_peered":"2024-09-16T11:43:31.779613+0000","last_unstale":"2024-09-16T11:43:55.687922+0000","last_undegraded":"2024-09-16T11:43:55.687922+0000","last_fullsized":"2024-09-16T11:43:55.687922+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:40.995663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000720899,"stat_sum":{"num_bytes":9002541,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1077,"num_read_kb":1961,"num_write":279,"num_write_kb":1703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"49'74","reported_seq":318,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T11:43:56.949029+0000","last_change":"2024-09-16T11:43:33.773914+0000","last_active":"2024-09-16T11:43:56.949029+0000","last_peered":"2024-09-16T11:43:56.949029+0000","last_clean":"2024-09-16T11:43:56.949029+0000","last_became_active":"2024-09-16T11:43:31.776132+0000","last_became_peered":"2024-09-16T11:43:31.776132+0000","last_unstale":"2024-09-16T11:43:56.949029+0000","last_undegraded":"2024-09-16T11:43:56.949029+0000","last_fullsized":"2024-09-16T11:43:56.949029+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:53.489198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023678600000000001,"stat_sum":{"num_bytes":1951413,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":586,"num_read_kb":636,"num_write":74,"num_write_kb":1378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":496905685,"num_objects":354,"num_object_clones":77,"num_object_copies":708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":354,"num_whiteouts":90,"num_read":35660,"num_read_kb":110436,"num_write":6164,"num_write_kb":80375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8314880,"data_stored":43148504,"data_compressed":3099396,"data_compressed_allocated":5632000,"data_compressed_original":41740562,"omap_allocated":0,"internal_metadata":0},"log_size":5041,"ondisk_log_size":5041,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303032,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068808,"statfs":{"total":96636764160,"available":96326459392,"internally_reserved":0,"allocated":1552384,"data_stored":5742870,"data_compressed":401445,"data_compressed_allocated":671744,"data_compressed_original":5320704,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]}]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303264,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068576,"statfs":{"total":96636764160,"available":96326221824,"internally_reserved":0,"allocated":1785856,"data_stored":6414656,"data_compressed":537415,"data_compressed_allocated":847872,"data_compressed_original":5967872,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542157,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311524,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94060316,"statfs":{"total":96636764160,"available":96317763584,"internally_reserved":0,"allocated":1720320,"data_stored":6415658,"data_compressed":395255,"data_compressed_allocated":802816,"data_compressed_original":5971383,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.17299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311696,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94060144,"statfs":{"total":96636764160,"available":96317587456,"internally_reserved":0,"allocated":1904640,"data_stored":6107167,"data_compressed":499732,"data_compressed_allocated":888832,"data_compressed_original":5610935,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.17999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.214}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1216512,"data_stored":4467668,"data_compressed":196588,"data_compressed_allocated":417792,"data_compressed_original":4101842,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308144,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063696,"statfs":{"total":96636764160,"available":96321224704,"internally_reserved":0,"allocated":2523136,"data_stored":14175673,"data_compressed":578466,"data_compressed_allocated":1429504,"data_compressed_original":13601900,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306548,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94065292,"statfs":{"total":96636764160,"available":96322859008,"internally_reserved":0,"allocated":888832,"data_stored":864915,"data_compressed":136398,"data_compressed_allocated":176128,"data_compressed_original":528384,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306916,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064924,"statfs":{"total":96636764160,"available":96322482176,"internally_reserved":0,"allocated":1261568,"data_stored":4317994,"data_compressed":286063,"data_compressed_allocated":532480,"data_compressed_original":3952640,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":790528,"data_stored":4820937,"data_compressed":297358,"data_compressed_allocated":614400,"data_compressed_original":4714496,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1388544,"data_stored":6121052,"data_compressed":514003,"data_compressed_allocated":917504,"data_compressed_original":5873079,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":629418,"data_compressed":136398,"data_compressed_allocated":176128,"data_compressed_original":528384,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1249280,"data_stored":6707329,"data_compressed":427147,"data_compressed_allocated":868352,"data_compressed_original":6507959,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1581056,"data_stored":8865531,"data_compressed":571079,"data_compressed_allocated":1077248,"data_compressed_original":8582866,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1224704,"data_stored":6159676,"data_compressed":537415,"data_compressed_allocated":847872,"data_compressed_original":5967872,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":729088,"data_stored":4668999,"data_compressed":202086,"data_compressed_allocated":458752,"data_compressed_original":4544210,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":983040,"data_stored":5175562,"data_compressed":413910,"data_compressed_allocated":671744,"data_compressed_original":5021696,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:44:00.421 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-16T11:44:00.421 INFO:teuthology.orchestra.run.smithi070.stdout:2982 punch from 0x1203bfc to 0x1206e6d, (0x3271 bytes) 2024-09-16T11:44:00.421 INFO:teuthology.orchestra.run.smithi070.stdout:2983 read 0xbd1037b thru 0xbd14877 (0x44fd bytes) 2024-09-16T11:44:00.421 INFO:teuthology.orchestra.run.smithi070.stdout:2984 write 0x96faf39 thru 0x96fdbca (0x2c92 bytes) 2024-09-16T11:44:00.422 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:00.413+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:00.424 INFO:teuthology.orchestra.run.smithi070.stdout:2985 trunc from 0xc214c7f to 0x606be76 2024-09-16T11:44:00.440 INFO:teuthology.orchestra.run.smithi070.stdout:2986 trunc from 0x606be76 to 0x2eea156 2024-09-16T11:44:00.459 INFO:teuthology.orchestra.run.smithi070.stdout:2987 writesame 0x496c839 thru 0x49772e4 data_size 0xc(0xaaac bytes) 2024-09-16T11:44:00.459 INFO:teuthology.orchestra.run.smithi070.stdout:2989 read 0x1698a07 thru 0x16a6360 (0xd95a bytes) 2024-09-16T11:44:00.459 INFO:teuthology.orchestra.run.smithi070.stdout:2990 read 0x25e3e80 thru 0x25f11ca (0xd34b bytes) 2024-09-16T11:44:00.460 INFO:teuthology.orchestra.run.smithi070.stdout:2991 punch from 0x50f12c to 0x516b73, (0x7a47 bytes) 2024-09-16T11:44:00.460 INFO:teuthology.orchestra.run.smithi070.stdout:2993 punch from 0x11d80a3 to 0x11da8d1, (0x282e bytes) 2024-09-16T11:44:00.473 INFO:teuthology.orchestra.run.smithi070.stdout:2994 write 0xb51a8a4 thru 0xb5255e9 (0xad46 bytes) 2024-09-16T11:44:00.477 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:00.478 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:00.478 INFO:teuthology.orchestra.run.smithi070.stdout:2998 read 0x561ec0f thru 0x562dc34 (0xf026 bytes) 2024-09-16T11:44:00.478 INFO:teuthology.orchestra.run.smithi070.stdout:2999 read 0x6b3b157 thru 0x6b3da22 (0x28cc bytes) 2024-09-16T11:44:00.478 INFO:teuthology.orchestra.run.smithi070.stdout:3003 read 0x6536bcf thru 0x653d206 (0x6638 bytes) 2024-09-16T11:44:00.484 INFO:teuthology.orchestra.run.smithi070.stdout:3004 write 0xda006e5 thru 0xda03f37 (0x3853 bytes) 2024-09-16T11:44:00.486 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:00.486 INFO:teuthology.orchestra.run.smithi070.stdout:3006 read 0x839235c thru 0x83989ae (0x6653 bytes) 2024-09-16T11:44:00.487 INFO:teuthology.orchestra.run.smithi070.stdout:3009 read 0x558d530 thru 0x5595d05 (0x87d6 bytes) 2024-09-16T11:44:00.487 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:00.487 INFO:teuthology.orchestra.run.smithi070.stdout:3012 write 0x1cc96fc thru 0x1ccdd5f (0x4664 bytes) 2024-09-16T11:44:00.490 INFO:teuthology.orchestra.run.smithi070.stdout:3015 read 0x8c4a13e thru 0x8c4dea7 (0x3d6a bytes) 2024-09-16T11:44:00.490 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:00.490 INFO:teuthology.orchestra.run.smithi070.stdout:3018 write 0x133baa thru 0x13a51c (0x6973 bytes) 2024-09-16T11:44:00.514 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:00.512+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:00.518 INFO:teuthology.orchestra.run.smithi070.stdout:3019 write 0xbb3cbd4 thru 0xbb3f554 (0x2981 bytes) 2024-09-16T11:44:00.521 INFO:teuthology.orchestra.run.smithi070.stdout:3020 punch from 0x708f919 to 0x70973f5, (0x7adc bytes) 2024-09-16T11:44:00.521 INFO:teuthology.orchestra.run.smithi070.stdout:3021 write 0xc6b39e7 thru 0xc6bdccf (0xa2e9 bytes) 2024-09-16T11:44:00.524 INFO:teuthology.orchestra.run.smithi070.stdout:3026 read 0x4c73779 thru 0x4c827fa (0xf082 bytes) 2024-09-16T11:44:00.524 INFO:teuthology.orchestra.run.smithi070.stdout:3027 write 0xf77422 thru 0xf7cc1e (0x57fd bytes) 2024-09-16T11:44:00.526 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:00.526 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:00.530 INFO:teuthology.orchestra.run.smithi070.stdout:3028 read 0x2055456 thru 0x2064f2a (0xfad5 bytes) 2024-09-16T11:44:00.530 INFO:teuthology.orchestra.run.smithi070.stdout:3030 read 0x511bbda thru 0x5126094 (0xa4bb bytes) 2024-09-16T11:44:00.530 INFO:teuthology.orchestra.run.smithi070.stdout:3031 punch from 0x37f00d5 to 0x37f5898, (0x57c3 bytes) 2024-09-16T11:44:00.530 INFO:teuthology.orchestra.run.smithi070.stdout:3034 read 0x6efbd55 thru 0x6f0124b (0x54f7 bytes) 2024-09-16T11:44:00.530 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:00.531 INFO:teuthology.orchestra.run.smithi070.stdout:3037 read 0x273f23d thru 0x27484a5 (0x9269 bytes) 2024-09-16T11:44:00.531 INFO:teuthology.orchestra.run.smithi070.stdout:3038 punch from 0xd31484a to 0xd31fbfa, (0xb3b0 bytes) 2024-09-16T11:44:00.531 INFO:teuthology.orchestra.run.smithi070.stdout:3039 read 0xa1ba4da thru 0xa1be703 (0x422a bytes) 2024-09-16T11:44:00.531 INFO:teuthology.orchestra.run.smithi070.stdout:3041 read 0x214216a thru 0x2148dea (0x6c81 bytes) 2024-09-16T11:44:00.531 INFO:teuthology.orchestra.run.smithi070.stdout:3042 write 0x4811402 thru 0x4821098 (0xfc97 bytes) 2024-09-16T11:44:00.533 INFO:teuthology.orchestra.run.smithi070.stdout:3043 read 0x3b18437 thru 0x3b1f7a1 (0x736b bytes) 2024-09-16T11:44:00.533 INFO:teuthology.orchestra.run.smithi070.stdout:3046 write 0xb6a61a5 thru 0xb6ada77 (0x78d3 bytes) 2024-09-16T11:44:00.535 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:00.536 INFO:teuthology.orchestra.run.smithi070.stdout:3047 read 0x33e2348 thru 0x33f0abd (0xe776 bytes) 2024-09-16T11:44:00.544 INFO:teuthology.orchestra.run.smithi070.stdout:3048 trunc from 0xda03f38 to 0x6135d50 2024-09-16T11:44:00.559 INFO:teuthology.orchestra.run.smithi070.stdout:3049 write 0x69e5814 thru 0x69ef48e (0x9c7b bytes) 2024-09-16T11:44:00.562 INFO:teuthology.orchestra.run.smithi070.stdout:3050 punch from 0x2677c83 to 0x267cf6f, (0x52ec bytes) 2024-09-16T11:44:00.562 INFO:teuthology.orchestra.run.smithi070.stdout:3052 punch from 0x1c5e433 to 0x1c5ff0f, (0x1adc bytes) 2024-09-16T11:44:00.562 INFO:teuthology.orchestra.run.smithi070.stdout:3056 read 0x5b97753 thru 0x5b9955d (0x1e0b bytes) 2024-09-16T11:44:00.562 INFO:teuthology.orchestra.run.smithi070.stdout:3057 punch from 0x4e58362 to 0x4e5fbaf, (0x784d bytes) 2024-09-16T11:44:00.565 INFO:teuthology.orchestra.run.smithi070.stdout:3059 trunc from 0x69ef48f to 0x9aa0cb8 2024-09-16T11:44:00.571 INFO:teuthology.orchestra.run.smithi070.stdout:3060 read 0x451e757 thru 0x451fb41 (0x13eb bytes) 2024-09-16T11:44:00.571 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:00.571 INFO:teuthology.orchestra.run.smithi070.stdout:3065 clone 19 order 22 su 16384 sc 8 2024-09-16T11:44:00.619 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:00.617+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:00.673 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:00.673 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:00.674 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:00.674 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:00.682 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:00.723 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:00.721+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:00.816 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:00.814+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:00.821 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:00.821 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:00.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:00.821 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:00.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.821 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.821 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4572853979999998, 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:00.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:00.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:44:00.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:44:00.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:44:00.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:44:00.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:44:00.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:44:00.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:44:00.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4572633689999996, 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.826 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:00.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:44:00.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:44:00.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:44:00.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:44:00.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:44:00.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:44:00.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:44:00.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:44:00.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:44:00.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:44:00.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4572486040000001, 2024-09-16T11:44:00.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:44:00.831 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.831 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.831 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.831 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.831 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.831 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:44:00.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.831 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:00.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:00.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:00.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:44:00.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:44:00.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:44:00.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:44:00.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:44:00.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:44:00.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:44:00.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:44:00.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:44:00.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:44:00.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:44:00.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:44:00.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:44:00.835 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.835 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.835 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.835 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.835 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:00.835 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4572285359999997, 2024-09-16T11:44:00.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:44:00.836 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.836 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.836 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:44:00.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.836 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:00.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:44:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:44:00.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:44:00.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:44:00.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:44:00.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:44:00.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:44:00.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:44:00.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:44:00.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:44:00.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:44:00.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:44:00.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:44:00.840 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.840 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.840 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.840 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.840 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:00.840 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4571977, 2024-09-16T11:44:00.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:44:00.840 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.840 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:00.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:00.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:44:00.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:44:00.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:44:00.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.844 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4571781799999997, 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:44:00.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:44:00.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:44:00.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:44:00.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:44:00.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:44:00.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:44:00.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:44:00.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:44:00.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:44:00.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:44:00.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:44:00.849 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4571506010000004, 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.850 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:44:00.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.851 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:00.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:00.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:00.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:44:00.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:44:00.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:44:00.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:44:00.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:44:00.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:44:00.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:44:00.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:44:00.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:44:00.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:44:00.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:44:00.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:44:00.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:44:00.854 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.854 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.854 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.854 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.855 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:00.855 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4571319059999999, 2024-09-16T11:44:00.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:44:00.855 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.855 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.855 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:44:00.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.855 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:44:00.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:44:00.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:44:00.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:44:00.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:44:00.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:44:00.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:44:00.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:44:00.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:44:00.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:44:00.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4571168969999997, 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.860 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:44:00.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.860 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:00.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:00.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:44:00.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:44:00.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:44:00.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:44:00.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:44:00.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:44:00.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.863 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4570847990000004, 2024-09-16T11:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:00.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:00.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:44:00.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:44:00.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:44:00.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:44:00.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:44:00.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:44:00.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:44:00.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:44:00.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:44:00.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:44:00.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:44:00.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:44:00.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4570611800000002, 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:44:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.869 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.869 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:44:00.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.869 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:00.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:44:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:44:00.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:44:00.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:44:00.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:44:00.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:44:00.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:44:00.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:44:00.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:44:00.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:44:00.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4570464510000001, 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.874 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.874 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:44:00.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.874 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:00.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:00.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:44:00.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:44:00.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:44:00.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:44:00.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:44:00.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:44:00.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.877 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.878 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:00.878 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.457024627, 2024-09-16T11:44:00.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:44:00.878 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.878 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.878 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:44:00.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.878 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:00.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:00.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:44:00.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:44:00.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:00.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:44:00.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:44:00.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:44:00.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:44:00.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:44:00.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:44:00.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.884 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.884 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.884 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:00.885 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4570099770000002, 2024-09-16T11:44:00.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:44:00.885 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.885 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.885 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:44:00.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.885 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:00.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:00.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:44:00.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:44:00.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:44:00.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:44:00.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:44:00.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:44:00.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:44:00.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:44:00.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:44:00.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:44:00.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:44:00.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:44:00.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:44:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:44:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4569884039999996, 2024-09-16T11:44:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:44:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.890 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.890 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:44:00.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.890 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:00.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:44:00.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:44:00.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:44:00.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:44:00.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:44:00.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:44:00.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:44:00.894 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.894 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.894 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.894 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.894 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:00.894 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.456968958, 2024-09-16T11:44:00.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:44:00.894 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.894 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.894 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.894 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.895 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.895 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:44:00.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.895 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:00.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4569322610000004, 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:00.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:44:00.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:44:00.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:44:00.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.902 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.903 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.903 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:00.903 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.456910315, 2024-09-16T11:44:00.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:44:00.903 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.903 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.903 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:44:00.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:00.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:44:00.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:44:00.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:00.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:44:00.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:44:00.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:44:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:44:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:44:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:44:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.4568884290000002, 2024-09-16T11:44:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:44:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:44:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:44:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:44:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:44:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:44:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:44:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:44:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:44:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:44:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.45687371, 2024-09-16T11:44:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:44:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:44:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:44:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:44:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:44:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:44:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:44:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:44:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:44:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:44:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:44:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:44:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:44:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:44:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:00.917 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:00.919 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T11:44:00.920 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:00.920 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:00.917+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:00.983 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone18 from 0x9aa0cb8 (overlap 0x2eea156) to 0x0 2024-09-16T11:44:01.019 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:01.018+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:01.073 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:01.073 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:01.073 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:01.073 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:01.082 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:01.125 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:01.123+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:01.166 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-16T11:44:01.185 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:01.185 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:01.185 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:01.185 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:01.185 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:01.195 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:01.220 INFO:teuthology.orchestra.run.smithi070.stdout:3066 punch from 0x82dad9d to 0x82e1bd6, (0x6e39 bytes) 2024-09-16T11:44:01.221 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:01.219+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:01.224 INFO:teuthology.orchestra.run.smithi070.stdout:3068 writesame 0x5d24cde thru 0x5d2bbc4 data_size 0x59(0x6ee7 bytes) 2024-09-16T11:44:01.236 INFO:teuthology.orchestra.run.smithi070.stdout:3069 writesame 0xd6236 thru 0xe0557 data_size 0x85(0xa322 bytes) 2024-09-16T11:44:01.285 INFO:teuthology.orchestra.run.smithi070.stdout:3070 write 0x9582b70 thru 0x958dd72 (0xb203 bytes) 2024-09-16T11:44:01.295 INFO:teuthology.orchestra.run.smithi070.stdout:3072 trunc from 0x9aa0cb8 to 0x84aa87f 2024-09-16T11:44:01.306 INFO:teuthology.orchestra.run.smithi070.stdout:3073 write 0x73e5a42 thru 0x73f1be4 (0xc1a3 bytes) 2024-09-16T11:44:01.321 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:01.319+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:01.326 INFO:teuthology.orchestra.run.smithi070.stdout:3075 write 0x24590ed thru 0x2468295 (0xf1a9 bytes) 2024-09-16T11:44:01.357 INFO:teuthology.orchestra.run.smithi070.stdout:3076 punch from 0x767f81f to 0x768f58c, (0xfd6d bytes) 2024-09-16T11:44:01.357 INFO:teuthology.orchestra.run.smithi070.stdout:3078 write 0x47d95da thru 0x47e0a0e (0x7435 bytes) 2024-09-16T11:44:01.358 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19942 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1990656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8298361549999997, 2024-09-16T11:44:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027866354999999999, 2024-09-16T11:44:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19942 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4899999999999995e-07 2024-09-16T11:44:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526281+0000", 2024-09-16T11:44:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-16T11:44:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526287+0000", 2024-09-16T11:44:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2199999999999997e-06 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526291+0000", 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554068+0000", 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027777076000000001 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554104+0000", 2024-09-16T11:44:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5537000000000002e-05 2024-09-16T11:44:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554145+0000", 2024-09-16T11:44:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1350000000000002e-05 2024-09-16T11:44:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19943 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2416640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.829819852, 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027947257999999999, 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19943 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5999999999999999e-07 2024-09-16T11:44:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526297+0000", 2024-09-16T11:44:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8059999999999999e-06 2024-09-16T11:44:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526303+0000", 2024-09-16T11:44:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6579999999999999e-06 2024-09-16T11:44:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526306+0000", 2024-09-16T11:44:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3969999999999998e-06 2024-09-16T11:44:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554165+0000", 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027858519000000002 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554201+0000", 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5908000000000002e-05 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554242+0000", 2024-09-16T11:44:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1510000000000001e-05 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19944 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2842624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8298049570000003, 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028052639000000001, 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19944 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526312+0000", 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526317+0000", 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5910000000000003e-06 2024-09-16T11:44:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526321+0000", 2024-09-16T11:44:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2339999999999999e-06 2024-09-16T11:44:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554263+0000", 2024-09-16T11:44:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027942181 2024-09-16T11:44:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554307+0000", 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4499999999999997e-05 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554362+0000", 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5078999999999999e-05 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19945 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3162112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8297904020000004, 2024-09-16T11:44:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028145673999999999, 2024-09-16T11:44:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19945 2024-09-16T11:44:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526325+0000", 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526326+0000", 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.733e-06 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526332+0000", 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526336+0000", 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8410000000000002e-06 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554389+0000", 2024-09-16T11:44:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028053576 2024-09-16T11:44:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554427+0000", 2024-09-16T11:44:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7299999999999999e-05 2024-09-16T11:44:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554470+0000", 2024-09-16T11:44:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3195000000000002e-05 2024-09-16T11:44:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19946 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3268608~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8297748230000002, 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028232587, 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19946 2024-09-16T11:44:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526346+0000", 2024-09-16T11:44:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6459999999999998e-06 2024-09-16T11:44:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526352+0000", 2024-09-16T11:44:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1070000000000001e-06 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526355+0000", 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554493+0000", 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.02813746 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554530+0000", 2024-09-16T11:44:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6881999999999997e-05 2024-09-16T11:44:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554572+0000", 2024-09-16T11:44:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2688999999999997e-05 2024-09-16T11:44:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19947 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4014080~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8297554690000002, 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028315646999999999, 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19947 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526360+0000", 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.01e-07 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526361+0000", 2024-09-16T11:44:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8300000000000001e-06 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526369+0000", 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526377+0000", 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0909999999999993e-06 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554593+0000", 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028216657999999999 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554632+0000", 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8828000000000002e-05 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554675+0000", 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2710000000000003e-05 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19948 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3588096~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8297330699999996, 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028394545, 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19948 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9200000000000001e-07 2024-09-16T11:44:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526384+0000", 2024-09-16T11:44:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:44:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526391+0000", 2024-09-16T11:44:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5150000000000003e-06 2024-09-16T11:44:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526394+0000", 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4240000000000002e-06 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554695+0000", 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028301218999999999 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554731+0000", 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6298999999999999e-05 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554776+0000", 2024-09-16T11:44:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4675999999999997e-05 2024-09-16T11:44:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19949 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1351680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8297167979999998, 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028490081, 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19949 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:44:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526400+0000", 2024-09-16T11:44:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7099999999999999e-06 2024-09-16T11:44:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526406+0000", 2024-09-16T11:44:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5140000000000001e-06 2024-09-16T11:44:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526409+0000", 2024-09-16T11:44:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5360000000000001e-06 2024-09-16T11:44:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554801+0000", 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028392378999999999 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554843+0000", 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1056000000000001e-05 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554888+0000", 2024-09-16T11:44:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466999999999997e-05 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19950 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1777664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8297018019999998, 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028661844999999998, 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19950 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526415+0000", 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.776e-06 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526422+0000", 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6930000000000001e-06 2024-09-16T11:44:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526425+0000", 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554923+0000", 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028498102000000001 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555000+0000", 2024-09-16T11:44:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6619999999999995e-05 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555075+0000", 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4931999999999993e-05 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19951 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2097152~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8296859909999998, 2024-09-16T11:44:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028755718, 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19951 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-16T11:44:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526431+0000", 2024-09-16T11:44:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526441+0000", 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7790000000000007e-06 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526445+0000", 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.634e-06 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555100+0000", 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654722000000001 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555138+0000", 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7996e-05 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555184+0000", 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6427999999999997e-05 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19952 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2203648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8296654310000005, 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028901157, 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19952 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526450+0000", 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526452+0000", 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9039999999999999e-06 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526465+0000", 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3559e-05 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526470+0000", 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555217+0000", 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028747287999999999 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555272+0000", 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4885000000000001e-05 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555350+0000", 2024-09-16T11:44:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8428000000000002e-05 2024-09-16T11:44:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19953 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2523136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8296408389999996, 2024-09-16T11:44:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029004309999999998, 2024-09-16T11:44:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19953 2024-09-16T11:44:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526476+0000", 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9810000000000002e-06 2024-09-16T11:44:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526482+0000", 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526485+0000", 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2760000000000001e-06 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555383+0000", 2024-09-16T11:44:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028898113 2024-09-16T11:44:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555433+0000", 2024-09-16T11:44:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9753999999999998e-05 2024-09-16T11:44:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555478+0000", 2024-09-16T11:44:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5006999999999998e-05 2024-09-16T11:44:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19954 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2949120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8296258420000004, 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029165054999999999, 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19954 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:01.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:01.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1799999999999995e-07 2024-09-16T11:44:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526491+0000", 2024-09-16T11:44:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7320000000000001e-06 2024-09-16T11:44:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526497+0000", 2024-09-16T11:44:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6869999999999999e-06 2024-09-16T11:44:01.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526500+0000", 2024-09-16T11:44:01.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:44:01.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555513+0000", 2024-09-16T11:44:01.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029012626 2024-09-16T11:44:01.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555576+0000", 2024-09-16T11:44:01.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3288999999999997e-05 2024-09-16T11:44:01.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555654+0000", 2024-09-16T11:44:01.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7861999999999994e-05 2024-09-16T11:44:01.417 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.417 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.417 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.417 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19955 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3375104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8296065400000003, 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029241979000000001, 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19955 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526509+0000", 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526510+0000", 2024-09-16T11:44:01.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.844e-06 2024-09-16T11:44:01.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526516+0000", 2024-09-16T11:44:01.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9039999999999997e-06 2024-09-16T11:44:01.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526520+0000", 2024-09-16T11:44:01.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3639999999999999e-06 2024-09-16T11:44:01.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555669+0000", 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029149628 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555706+0000", 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7017000000000002e-05 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555750+0000", 2024-09-16T11:44:01.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3856999999999997e-05 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19956 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3694592~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8295915850000002, 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029398799999999999, 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.422 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19956 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526524+0000", 2024-09-16T11:44:01.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:44:01.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526525+0000", 2024-09-16T11:44:01.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-16T11:44:01.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526531+0000", 2024-09-16T11:44:01.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6010000000000002e-06 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526535+0000", 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.597e-06 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555785+0000", 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029250235999999999 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555845+0000", 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0393000000000003e-05 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555922+0000", 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6710000000000002e-05 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19957 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4120576~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:01.426 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8295764319999996, 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029481157000000001, 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19957 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526539+0000", 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526540+0000", 2024-09-16T11:44:01.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.759e-06 2024-09-16T11:44:01.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526546+0000", 2024-09-16T11:44:01.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1079999999999998e-06 2024-09-16T11:44:01.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526550+0000", 2024-09-16T11:44:01.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.269e-06 2024-09-16T11:44:01.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555939+0000", 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029389444000000001 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555977+0000", 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7487999999999998e-05 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.556019+0000", 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2756999999999997e-05 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.430 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:28959 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.34672002000000002, 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031145285000000002, 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 28959 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.431 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:01.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:01.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009396+0000", 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.296e-06 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030864+0000", 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.021468451 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030892+0000", 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7555999999999998e-05 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030902+0000", 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9550000000000004e-06 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030979+0000", 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7137000000000004e-05 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031051+0000", 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1475000000000003e-05 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031389+0000", 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033827100000000001 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.039264+0000", 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078751700000000008 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040484+0000", 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012199579999999999 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040517+0000", 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2960999999999998e-05 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040540+0000", 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3054999999999999e-05 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.436 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30605 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 3915776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.112413761, 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938655999999999, 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30605 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:01.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:01.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:01.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:01.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243704+0000", 2024-09-16T11:44:01.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2199999999999999e-06 2024-09-16T11:44:01.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243710+0000", 2024-09-16T11:44:01.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6340000000000003e-06 2024-09-16T11:44:01.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243714+0000", 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5030000000000002e-06 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271450+0000", 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027736035999999999 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271536+0000", 2024-09-16T11:44:01.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6521999999999996e-05 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271640+0000", 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103341 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30607 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4046848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.112390606, 2024-09-16T11:44:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028130446, 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30607 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243725+0000", 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2399999999999998e-07 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243727+0000", 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3379999999999999e-06 2024-09-16T11:44:01.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243733+0000", 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4280000000000001e-06 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243737+0000", 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8990000000000003e-06 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271676+0000", 2024-09-16T11:44:01.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938481000000001 2024-09-16T11:44:01.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271757+0000", 2024-09-16T11:44:01.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1396999999999993e-05 2024-09-16T11:44:01.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271855+0000", 2024-09-16T11:44:01.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7379000000000004e-05 2024-09-16T11:44:01.445 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.445 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.445 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30608 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.11237375500000001, 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028323879, 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30608 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.446 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:01.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:44:01.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243743+0000", 2024-09-16T11:44:01.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.762e-06 2024-09-16T11:44:01.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243749+0000", 2024-09-16T11:44:01.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5670000000000002e-06 2024-09-16T11:44:01.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243758+0000", 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8309999999999996e-06 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271879+0000", 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028120501999999999 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271965+0000", 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5755999999999997e-05 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.272065+0000", 2024-09-16T11:44:01.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000100101 2024-09-16T11:44:01.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.450 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:01.452 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:01.454 INFO:teuthology.orchestra.run.smithi070.stdout:3081 read 0x32b44b3 thru 0x32b9b50 (0x569e bytes) 2024-09-16T11:44:01.454 INFO:teuthology.orchestra.run.smithi070.stdout:3082 writesame 0xa0f3c48 thru 0xa0f4fcd data_size 0x6(0x1386 bytes) 2024-09-16T11:44:01.454 INFO:teuthology.orchestra.run.smithi070.stdout:3083 punch from 0x3fced67 to 0x3fd0a4e, (0x1ce7 bytes) 2024-09-16T11:44:01.454 INFO:teuthology.orchestra.run.smithi070.stdout:3084 write 0xa5ea6c thru 0xa60378 (0x190d bytes) 2024-09-16T11:44:01.454 INFO:teuthology.orchestra.run.smithi070.stdout:3085 trunc from 0xa0f4fce to 0xe0985d7 2024-09-16T11:44:01.454 INFO:teuthology.orchestra.run.smithi070.stdout:3086 read 0x1fc1918 thru 0x1fcf47d (0xdb66 bytes) 2024-09-16T11:44:01.454 INFO:teuthology.orchestra.run.smithi070.stdout:3088 read 0x829638b thru 0x829d821 (0x7497 bytes) 2024-09-16T11:44:01.454 INFO:teuthology.orchestra.run.smithi070.stdout:3089 writesame 0x1ced1f9 thru 0x1cfb8ff data_size 0x47(0xe707 bytes) 2024-09-16T11:44:01.454 INFO:teuthology.orchestra.run.smithi070.stdout:3090 read 0xd871d1a thru 0xd875437 (0x371e bytes) 2024-09-16T11:44:01.454 INFO:teuthology.orchestra.run.smithi070.stdout:3094 read 0xa868a72 thru 0xa86c865 (0x3df4 bytes) 2024-09-16T11:44:01.454 INFO:teuthology.orchestra.run.smithi070.stdout:3096 read 0xacdeaeb thru 0xace1cdc (0x31f2 bytes) 2024-09-16T11:44:01.455 INFO:teuthology.orchestra.run.smithi070.stdout:3097 read 0xb4e783c thru 0xb4eda61 (0x6226 bytes) 2024-09-16T11:44:01.455 INFO:teuthology.orchestra.run.smithi070.stdout:3098 trunc from 0xe0985d7 to 0xb916c83 2024-09-16T11:44:01.455 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.455 INFO:teuthology.orchestra.run.smithi070.stdout:3100 write 0xa2e14b9 thru 0xa2e5bf4 (0x473c bytes) 2024-09-16T11:44:01.455 INFO:teuthology.orchestra.run.smithi070.stdout:3101 writesame 0x484f58d thru 0x485839d data_size 0x9(0x8e11 bytes) 2024-09-16T11:44:01.455 INFO:teuthology.orchestra.run.smithi070.stdout:3102 punch from 0x53cedc4 to 0x53d52ed, (0x6529 bytes) 2024-09-16T11:44:01.455 INFO:teuthology.orchestra.run.smithi070.stdout:3105 read 0x4381f95 thru 0x4386541 (0x45ad bytes) 2024-09-16T11:44:01.455 INFO:teuthology.orchestra.run.smithi070.stdout:3106 read 0xa04f086 thru 0xa0585ca (0x9545 bytes) 2024-09-16T11:44:01.455 INFO:teuthology.orchestra.run.smithi070.stdout:3107 punch from 0x8045f0f to 0x80531bc, (0xd2ad bytes) 2024-09-16T11:44:01.455 INFO:teuthology.orchestra.run.smithi070.stdout:3108 write 0x9756448 thru 0x975e548 (0x8101 bytes) 2024-09-16T11:44:01.455 INFO:teuthology.orchestra.run.smithi070.stdout:3109 read 0x4487755 thru 0x4493464 (0xbd10 bytes) 2024-09-16T11:44:01.455 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:01.450+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:01.456 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.456 INFO:teuthology.orchestra.run.smithi070.stdout:3112 write 0x80eec8b thru 0x80fc395 (0xd70b bytes) 2024-09-16T11:44:01.456 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.458 INFO:teuthology.orchestra.run.smithi070.stdout:3114 writesame 0xc002ccb thru 0xc0073f3 data_size 0x1(0x4729 bytes) 2024-09-16T11:44:01.458 INFO:teuthology.orchestra.run.smithi070.stdout:3115 read 0x3ec5c05 thru 0x3ecda87 (0x7e83 bytes) 2024-09-16T11:44:01.458 INFO:teuthology.orchestra.run.smithi070.stdout:3116 read 0x6621f64 thru 0x66319ea (0xfa87 bytes) 2024-09-16T11:44:01.459 INFO:teuthology.orchestra.run.smithi070.stdout:3120 writesame 0x37f57c2 thru 0x3802c11 data_size 0x9e(0xd450 bytes) 2024-09-16T11:44:01.468 INFO:teuthology.orchestra.run.smithi070.stdout:3121 writesame 0xadceb98 thru 0xadd1347 data_size 0x14(0x27b0 bytes) 2024-09-16T11:44:01.471 INFO:teuthology.orchestra.run.smithi070.stdout:3122 punch from 0x6baa5b5 to 0x6bb6083, (0xbace bytes) 2024-09-16T11:44:01.472 INFO:teuthology.orchestra.run.smithi070.stdout:3124 read 0x3f4f0c thru 0x3fd26d (0x8362 bytes) 2024-09-16T11:44:01.472 INFO:teuthology.orchestra.run.smithi070.stdout:3125 write 0x150053a thru 0x1501647 (0x110e bytes) 2024-09-16T11:44:01.473 INFO:teuthology.orchestra.run.smithi070.stdout:3126 read 0x9d2155 thru 0x9d81ac (0x6058 bytes) 2024-09-16T11:44:01.474 INFO:teuthology.orchestra.run.smithi070.stdout:3127 writesame 0x65ad1b9 thru 0x65b39ff data_size 0x13(0x6847 bytes) 2024-09-16T11:44:01.476 INFO:teuthology.orchestra.run.smithi070.stdout:3128 write 0x171f1ab thru 0x1721475 (0x22cb bytes) 2024-09-16T11:44:01.477 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.477 INFO:teuthology.orchestra.run.smithi070.stdout:3130 punch from 0x82d98fb to 0x82e4d74, (0xb479 bytes) 2024-09-16T11:44:01.478 INFO:teuthology.orchestra.run.smithi070.stdout:3132 punch from 0x91fd289 to 0x9209a55, (0xc7cc bytes) 2024-09-16T11:44:01.478 INFO:teuthology.orchestra.run.smithi070.stdout:3133 read 0x6039c6c thru 0x603ec9a (0x502f bytes) 2024-09-16T11:44:01.485 INFO:teuthology.orchestra.run.smithi070.stdout:3136 writesame 0xdfe9c8c thru 0xdfec316 data_size 0x27(0x268b bytes) 2024-09-16T11:44:01.485 INFO:teuthology.orchestra.run.smithi070.stdout:3137 read 0x2f5c81c thru 0x2f638a7 (0x708c bytes) 2024-09-16T11:44:01.485 INFO:teuthology.orchestra.run.smithi070.stdout:3138 write 0x706f25e thru 0x7076630 (0x73d3 bytes) 2024-09-16T11:44:01.500 INFO:teuthology.orchestra.run.smithi070.stdout:3139 trunc from 0xdfec317 to 0x79af65d 2024-09-16T11:44:01.534 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.539 INFO:teuthology.orchestra.run.smithi070.stdout:3141 trunc from 0x79af65d to 0xb18df9c 2024-09-16T11:44:01.544 INFO:teuthology.orchestra.run.smithi070.stdout:3142 writesame 0x626d99c thru 0x626ecc6 data_size 0x1(0x132b bytes) 2024-09-16T11:44:01.552 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:01.551+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:01.553 INFO:teuthology.orchestra.run.smithi070.stdout:3144 writesame 0xc43e8ff thru 0xc4425d6 data_size 0x42(0x3cd8 bytes) 2024-09-16T11:44:01.560 INFO:teuthology.orchestra.run.smithi070.stdout:3149 writesame 0xce9c754 thru 0xcea5cbf data_size 0x83(0x956c bytes) 2024-09-16T11:44:01.560 INFO:teuthology.orchestra.run.smithi070.stdout:3150 punch from 0xaec0e70 to 0xaecc2da, (0xb46a bytes) 2024-09-16T11:44:01.560 INFO:teuthology.orchestra.run.smithi070.stdout:3151 write 0x14690f5 thru 0x147724f (0xe15b bytes) 2024-09-16T11:44:01.579 INFO:teuthology.orchestra.run.smithi070.stdout:3152 read 0x1bda125 thru 0x1be789e (0xd77a bytes) 2024-09-16T11:44:01.580 INFO:teuthology.orchestra.run.smithi070.stdout:3154 punch from 0x4c8acf to 0x4cf700, (0x6c31 bytes) 2024-09-16T11:44:01.580 INFO:teuthology.orchestra.run.smithi070.stdout:3155 write 0xa0c76eb thru 0xa0ce5fb (0x6f11 bytes) 2024-09-16T11:44:01.587 INFO:teuthology.orchestra.run.smithi070.stdout:3156 writesame 0xaab63f8 thru 0xaac32f5 data_size 0x23(0xcefe bytes) 2024-09-16T11:44:01.592 INFO:teuthology.orchestra.run.smithi070.stdout:3158 write 0x67294a4 thru 0x6732fb2 (0x9b0f bytes) 2024-09-16T11:44:01.594 INFO:teuthology.orchestra.run.smithi070.stdout:3160 read 0xa2a250 thru 0xa34c39 (0xa9ea bytes) 2024-09-16T11:44:01.595 INFO:teuthology.orchestra.run.smithi070.stdout:3161 read 0x9fd3e41 thru 0x9fd4709 (0x8c9 bytes) 2024-09-16T11:44:01.595 INFO:teuthology.orchestra.run.smithi070.stdout:3164 write 0x695322f thru 0x69563c9 (0x319b bytes) 2024-09-16T11:44:01.596 INFO:teuthology.orchestra.run.smithi070.stdout:3165 writesame 0x2b40cab thru 0x2b4a5a5 data_size 0x1(0x98fb bytes) 2024-09-16T11:44:01.603 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:01.603 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:01.603 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:01.603 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:01.606 INFO:teuthology.orchestra.run.smithi070.stdout:3166 write 0xc7521a3 thru 0xc75c283 (0xa0e1 bytes) 2024-09-16T11:44:01.613 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:01.625 INFO:teuthology.orchestra.run.smithi070.stdout:3167 trunc from 0xcea5cc0 to 0x1918558 2024-09-16T11:44:01.653 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:01.652+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:01.683 INFO:teuthology.orchestra.run.smithi070.stdout:3168 write 0xd543eb1 thru 0xd54a0e4 (0x6234 bytes) 2024-09-16T11:44:01.687 INFO:teuthology.orchestra.run.smithi070.stdout:3169 write 0x14ef252 thru 0x14fdd03 (0xeab2 bytes) 2024-09-16T11:44:01.689 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.689 INFO:teuthology.orchestra.run.smithi070.stdout:3171 writesame 0xa596b6 thru 0xa63067 data_size 0x67(0x99b2 bytes) 2024-09-16T11:44:01.690 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.692 INFO:teuthology.orchestra.run.smithi070.stdout:3173 trunc from 0xd54a0e5 to 0xe81fdc2 2024-09-16T11:44:01.700 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.700 INFO:teuthology.orchestra.run.smithi070.stdout:3175 read 0xa519de2 thru 0xa51be06 (0x2025 bytes) 2024-09-16T11:44:01.707 INFO:teuthology.orchestra.run.smithi070.stdout:3177 trunc from 0xe81fdc2 to 0x5eff48d 2024-09-16T11:44:01.721 INFO:teuthology.orchestra.run.smithi070.stdout:3179 trunc from 0x5eff48d to 0x8b4c14d 2024-09-16T11:44:01.724 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:01.724 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:01.725 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:01.725 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:01.725 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:01.725 INFO:teuthology.orchestra.run.smithi070.stdout:3180 read 0x51807bb thru 0x518c3a4 (0xbbea bytes) 2024-09-16T11:44:01.725 INFO:teuthology.orchestra.run.smithi070.stdout:3181 read 0x63d7991 thru 0x63da6e1 (0x2d51 bytes) 2024-09-16T11:44:01.725 INFO:teuthology.orchestra.run.smithi070.stdout:3182 writesame 0x6c6521 thru 0x6d194a data_size 0x3(0xb42a bytes) 2024-09-16T11:44:01.732 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.732 INFO:teuthology.orchestra.run.smithi070.stdout:3184 read 0x8452a0e thru 0x845780c (0x4dff bytes) 2024-09-16T11:44:01.733 INFO:teuthology.orchestra.run.smithi070.stdout:3187 writesame 0xbd1b61 thru 0xbd35b5 data_size 0x15(0x1a55 bytes) 2024-09-16T11:44:01.734 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.734 INFO:teuthology.orchestra.run.smithi070.stdout:3189 write 0x73d3905 thru 0x73d6568 (0x2c64 bytes) 2024-09-16T11:44:01.735 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:01.737 INFO:teuthology.orchestra.run.smithi070.stdout:3191 write 0x64349b0 thru 0x643bc4e (0x729f bytes) 2024-09-16T11:44:01.740 INFO:teuthology.orchestra.run.smithi070.stdout:3192 write 0x21a1de0 thru 0x21ad2be (0xb4df bytes) 2024-09-16T11:44:01.745 INFO:teuthology.orchestra.run.smithi070.stdout:3193 punch from 0x2a1214d to 0x2a216fb, (0xf5ae bytes) 2024-09-16T11:44:01.749 INFO:teuthology.orchestra.run.smithi070.stdout:3194 trunc from 0x8b4c14d to 0xc03cdef 2024-09-16T11:44:01.755 INFO:teuthology.orchestra.run.smithi070.stdout:3195 write 0x313d4ea thru 0x3149736 (0xc24d bytes) 2024-09-16T11:44:01.758 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.759 INFO:teuthology.orchestra.run.smithi070.stdout:3197 write 0x52eca2e thru 0x52fc68b (0xfc5e bytes) 2024-09-16T11:44:01.766 INFO:teuthology.orchestra.run.smithi070.stdout:3198 write 0x835968e thru 0x835cd17 (0x368a bytes) 2024-09-16T11:44:01.773 INFO:teuthology.orchestra.run.smithi070.stdout:3199 trunc from 0xc03cdef to 0xe990329 2024-09-16T11:44:01.777 INFO:teuthology.orchestra.run.smithi070.stdout:3200 punch from 0xc0daa2a to 0xc0e7b44, (0xd11a bytes) 2024-09-16T11:44:01.777 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.777 INFO:teuthology.orchestra.run.smithi070.stdout:3202 writesame 0x51d7fba thru 0x51e63a5 data_size 0x3a(0xe3ec bytes) 2024-09-16T11:44:01.778 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:01.776+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:01.781 INFO:teuthology.orchestra.run.smithi070.stdout:3203 read 0x2a9e946 thru 0x2aa13ed (0x2aa8 bytes) 2024-09-16T11:44:01.781 INFO:teuthology.orchestra.run.smithi070.stdout:3204 read 0x8851224 thru 0x885ae19 (0x9bf6 bytes) 2024-09-16T11:44:01.781 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.781 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.781 INFO:teuthology.orchestra.run.smithi070.stdout:3210 write 0xf33f8c thru 0xf39b7d (0x5bf2 bytes) 2024-09-16T11:44:01.783 INFO:teuthology.orchestra.run.smithi070.stdout:3211 writesame 0x499c0c1 thru 0x499fe1a data_size 0x1(0x3d5a bytes) 2024-09-16T11:44:01.790 INFO:teuthology.orchestra.run.smithi070.stdout:3213 read 0xb7f21eb thru 0xb7f2e18 (0xc2e bytes) 2024-09-16T11:44:01.790 INFO:teuthology.orchestra.run.smithi070.stdout:3214 read 0x18d3004 thru 0x18d45e5 (0x15e2 bytes) 2024-09-16T11:44:01.791 INFO:teuthology.orchestra.run.smithi070.stdout:3216 punch from 0x466d32b to 0x4672757, (0x542c bytes) 2024-09-16T11:44:01.791 INFO:teuthology.orchestra.run.smithi070.stdout:3217 read 0xda74fd thru 0xdb55b5 (0xe0b9 bytes) 2024-09-16T11:44:01.791 INFO:teuthology.orchestra.run.smithi070.stdout:3219 write 0xa638199 thru 0xa63d2eb (0x5153 bytes) 2024-09-16T11:44:01.795 INFO:teuthology.orchestra.run.smithi070.stdout:3220 punch from 0x322871c to 0x3231a9b, (0x937f bytes) 2024-09-16T11:44:01.795 INFO:teuthology.orchestra.run.smithi070.stdout:3221 writesame 0x8624548 thru 0x86284ac data_size 0x1(0x3f65 bytes) 2024-09-16T11:44:01.803 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.803 INFO:teuthology.orchestra.run.smithi070.stdout:3223 read 0xdc7c48e thru 0xdc85c4f (0x97c2 bytes) 2024-09-16T11:44:01.803 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.803 INFO:teuthology.orchestra.run.smithi070.stdout:3226 read 0x6b96fcb thru 0x6b9fdda (0x8e10 bytes) 2024-09-16T11:44:01.804 INFO:teuthology.orchestra.run.smithi070.stdout:3227 writesame 0x5677445 thru 0x5683027 data_size 0x1(0xbbe3 bytes) 2024-09-16T11:44:01.825 INFO:teuthology.orchestra.run.smithi070.stdout:3228 trunc from 0xe990329 to 0x6b287dc 2024-09-16T11:44:01.836 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.836 INFO:teuthology.orchestra.run.smithi070.stdout:3232 punch from 0x5510965 to 0x551834d, (0x79e8 bytes) 2024-09-16T11:44:01.838 INFO:teuthology.orchestra.run.smithi070.stdout:3234 trunc from 0x6b287dc to 0x4970f78 2024-09-16T11:44:01.842 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:9179 4.5 4:a03f235d:::rbd_data.11c1dc795e15.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:44:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:46.201943+0000", 2024-09-16T11:44:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 15.637387336, 2024-09-16T11:44:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028095163999999999, 2024-09-16T11:44:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 9179 2024-09-16T11:44:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201943+0000", 2024-09-16T11:44:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201943+0000", 2024-09-16T11:44:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201944+0000", 2024-09-16T11:44:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.283e-06 2024-09-16T11:44:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201948+0000", 2024-09-16T11:44:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0210000000000001e-06 2024-09-16T11:44:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201960+0000", 2024-09-16T11:44:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1420000000000001e-05 2024-09-16T11:44:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201965+0000", 2024-09-16T11:44:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6570000000000002e-06 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201998+0000", 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2882000000000001e-05 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.202046+0000", 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7274000000000002e-05 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.206571+0000", 2024-09-16T11:44:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045255080000000001 2024-09-16T11:44:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.229964+0000", 2024-09-16T11:44:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023392901000000001 2024-09-16T11:44:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.229973+0000", 2024-09-16T11:44:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8589999999999994e-06 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.230010+0000", 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7261000000000001e-05 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.230038+0000", 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8098000000000001e-05 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19271 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7766016~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472355+0000", 2024-09-16T11:44:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.3669752229999999, 2024-09-16T11:44:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028037484000000001, 2024-09-16T11:44:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19271 2024-09-16T11:44:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472355+0000", 2024-09-16T11:44:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472355+0000", 2024-09-16T11:44:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472356+0000", 2024-09-16T11:44:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:44:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472357+0000", 2024-09-16T11:44:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7010000000000001e-06 2024-09-16T11:44:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472363+0000", 2024-09-16T11:44:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5210000000000002e-06 2024-09-16T11:44:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472366+0000", 2024-09-16T11:44:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.444e-06 2024-09-16T11:44:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500244+0000", 2024-09-16T11:44:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027878028999999999 2024-09-16T11:44:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500320+0000", 2024-09-16T11:44:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5443000000000003e-05 2024-09-16T11:44:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500393+0000", 2024-09-16T11:44:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2849999999999995e-05 2024-09-16T11:44:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19272 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 8192000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472377+0000", 2024-09-16T11:44:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.3669532960000002, 2024-09-16T11:44:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028126168999999999, 2024-09-16T11:44:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19272 2024-09-16T11:44:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472377+0000", 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472377+0000", 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472378+0000", 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472379+0000", 2024-09-16T11:44:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8449999999999999e-06 2024-09-16T11:44:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472386+0000", 2024-09-16T11:44:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5549999999999999e-06 2024-09-16T11:44:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472390+0000", 2024-09-16T11:44:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:44:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500422+0000", 2024-09-16T11:44:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028032715 2024-09-16T11:44:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500468+0000", 2024-09-16T11:44:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5411000000000003e-05 2024-09-16T11:44:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500503+0000", 2024-09-16T11:44:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-05 2024-09-16T11:44:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19274 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 5529600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472427+0000", 2024-09-16T11:44:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.3669033380000002, 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028248603000000001, 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19274 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472427+0000", 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472427+0000", 2024-09-16T11:44:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472428+0000", 2024-09-16T11:44:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4600000000000005e-07 2024-09-16T11:44:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472429+0000", 2024-09-16T11:44:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:44:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472436+0000", 2024-09-16T11:44:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1340000000000001e-06 2024-09-16T11:44:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472444+0000", 2024-09-16T11:44:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9299999999999992e-06 2024-09-16T11:44:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500532+0000", 2024-09-16T11:44:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028087903000000001 2024-09-16T11:44:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500611+0000", 2024-09-16T11:44:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9048999999999995e-05 2024-09-16T11:44:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500676+0000", 2024-09-16T11:44:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4270000000000006e-05 2024-09-16T11:44:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19275 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 5955584~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472468+0000", 2024-09-16T11:44:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.3668624649999996, 2024-09-16T11:44:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028317512, 2024-09-16T11:44:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19275 2024-09-16T11:44:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472468+0000", 2024-09-16T11:44:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472468+0000", 2024-09-16T11:44:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472469+0000", 2024-09-16T11:44:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9800000000000003e-07 2024-09-16T11:44:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472472+0000", 2024-09-16T11:44:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-06 2024-09-16T11:44:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472483+0000", 2024-09-16T11:44:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0883000000000001e-05 2024-09-16T11:44:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472497+0000", 2024-09-16T11:44:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4158999999999999e-05 2024-09-16T11:44:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500703+0000", 2024-09-16T11:44:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028205480000000002 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500750+0000", 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6978999999999999e-05 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500785+0000", 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5602999999999998e-05 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19276 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 6381568~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472519+0000", 2024-09-16T11:44:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.3668116470000005, 2024-09-16T11:44:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028436959000000001, 2024-09-16T11:44:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19276 2024-09-16T11:44:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.869 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472519+0000", 2024-09-16T11:44:01.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472519+0000", 2024-09-16T11:44:01.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472519+0000", 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472526+0000", 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7549999999999997e-06 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472534+0000", 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5700000000000004e-06 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472538+0000", 2024-09-16T11:44:01.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0239999999999999e-06 2024-09-16T11:44:01.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500815+0000", 2024-09-16T11:44:01.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028276881 2024-09-16T11:44:01.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500890+0000", 2024-09-16T11:44:01.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5693999999999997e-05 2024-09-16T11:44:01.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500956+0000", 2024-09-16T11:44:01.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5392e-05 2024-09-16T11:44:01.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.872 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.872 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19278 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 6701056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.872 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472568+0000", 2024-09-16T11:44:01.873 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.3667623769999997, 2024-09-16T11:44:01.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028494954999999999, 2024-09-16T11:44:01.873 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.873 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.873 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19278 2024-09-16T11:44:01.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.873 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472568+0000", 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472568+0000", 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472569+0000", 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-16T11:44:01.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472571+0000", 2024-09-16T11:44:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9850000000000001e-06 2024-09-16T11:44:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472577+0000", 2024-09-16T11:44:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0549999999999996e-06 2024-09-16T11:44:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472582+0000", 2024-09-16T11:44:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-16T11:44:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500982+0000", 2024-09-16T11:44:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028399974000000001 2024-09-16T11:44:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501026+0000", 2024-09-16T11:44:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4440000000000001e-05 2024-09-16T11:44:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501063+0000", 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6999999999999998e-05 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19281 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472740+0000", 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.366590371, 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028491267000000001, 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.877 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19281 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472740+0000", 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472740+0000", 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472741+0000", 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0699999999999996e-07 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472745+0000", 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472754+0000", 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9220000000000003e-06 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472759+0000", 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0629999999999996e-06 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501091+0000", 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028332053999999999 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501168+0000", 2024-09-16T11:44:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7033000000000006e-05 2024-09-16T11:44:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501231+0000", 2024-09-16T11:44:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3096999999999996e-05 2024-09-16T11:44:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.881 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19282 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.3664932390000004, 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521151000000002, 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19282 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472838+0000", 2024-09-16T11:44:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:44:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472848+0000", 2024-09-16T11:44:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9799999999999993e-06 2024-09-16T11:44:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472858+0000", 2024-09-16T11:44:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9429999999999995e-06 2024-09-16T11:44:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472862+0000", 2024-09-16T11:44:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4460000000000003e-06 2024-09-16T11:44:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501277+0000", 2024-09-16T11:44:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028414900999999999 2024-09-16T11:44:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501322+0000", 2024-09-16T11:44:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4880999999999999e-05 2024-09-16T11:44:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501358+0000", 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6245000000000003e-05 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19284 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472990+0000", 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.3663402759999999, 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028479695999999999, 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.886 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19284 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472990+0000", 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472990+0000", 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472991+0000", 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472994+0000", 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1219999999999999e-06 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473001+0000", 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4719999999999998e-06 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473006+0000", 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5040000000000004e-06 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501388+0000", 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381661999999998 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501432+0000", 2024-09-16T11:44:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4131999999999997e-05 2024-09-16T11:44:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501470+0000", 2024-09-16T11:44:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8114000000000001e-05 2024-09-16T11:44:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19285 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7872512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473074+0000", 2024-09-16T11:44:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.3662560370000003, 2024-09-16T11:44:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028486999999999998, 2024-09-16T11:44:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19285 2024-09-16T11:44:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.891 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473074+0000", 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473074+0000", 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473075+0000", 2024-09-16T11:44:01.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473078+0000", 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5679999999999998e-06 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473083+0000", 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1930000000000001e-06 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473091+0000", 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5639999999999997e-06 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501490+0000", 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028398685 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501527+0000", 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7426999999999999e-05 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501561+0000", 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3994e-05 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19286 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7979008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.3661203070000001, 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521860999999999, 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.895 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19286 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473211+0000", 2024-09-16T11:44:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:44:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473214+0000", 2024-09-16T11:44:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1080000000000001e-06 2024-09-16T11:44:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473226+0000", 2024-09-16T11:44:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2441e-05 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473234+0000", 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729999999999998e-06 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501592+0000", 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028358491999999999 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501671+0000", 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8846999999999999e-05 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501732+0000", 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1118000000000007e-05 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19288 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 8298496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473367+0000", 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.3659630700000003, 2024-09-16T11:44:01.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028460488999999999, 2024-09-16T11:44:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19288 2024-09-16T11:44:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473367+0000", 2024-09-16T11:44:01.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473367+0000", 2024-09-16T11:44:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473368+0000", 2024-09-16T11:44:01.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:44:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473371+0000", 2024-09-16T11:44:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1540000000000002e-06 2024-09-16T11:44:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473386+0000", 2024-09-16T11:44:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4430000000000001e-05 2024-09-16T11:44:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473390+0000", 2024-09-16T11:44:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999996e-06 2024-09-16T11:44:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501754+0000", 2024-09-16T11:44:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028363546999999999 2024-09-16T11:44:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501794+0000", 2024-09-16T11:44:01.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9965000000000001e-05 2024-09-16T11:44:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501828+0000", 2024-09-16T11:44:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3926e-05 2024-09-16T11:44:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20616 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 6545408~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560522+0000", 2024-09-16T11:44:01.904 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.2788081519999999, 2024-09-16T11:44:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028053345, 2024-09-16T11:44:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20616 2024-09-16T11:44:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560522+0000", 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560522+0000", 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560522+0000", 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560523+0000", 2024-09-16T11:44:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-16T11:44:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560526+0000", 2024-09-16T11:44:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7130000000000001e-06 2024-09-16T11:44:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560528+0000", 2024-09-16T11:44:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:44:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588461+0000", 2024-09-16T11:44:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027932413999999999 2024-09-16T11:44:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588505+0000", 2024-09-16T11:44:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4199000000000002e-05 2024-09-16T11:44:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588576+0000", 2024-09-16T11:44:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0899e-05 2024-09-16T11:44:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20617 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 6864896~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560531+0000", 2024-09-16T11:44:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.278799459, 2024-09-16T11:44:01.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028225698, 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20617 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560531+0000", 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560531+0000", 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560531+0000", 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560532+0000", 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7900000000000007e-07 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560535+0000", 2024-09-16T11:44:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7539999999999998e-06 2024-09-16T11:44:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560537+0000", 2024-09-16T11:44:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7349999999999999e-06 2024-09-16T11:44:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588606+0000", 2024-09-16T11:44:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028069436999999999 2024-09-16T11:44:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588668+0000", 2024-09-16T11:44:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2287999999999997e-05 2024-09-16T11:44:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588757+0000", 2024-09-16T11:44:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8299000000000003e-05 2024-09-16T11:44:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.913 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20618 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7290880~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560539+0000", 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.2787915309999995, 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028393137999999998, 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20618 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.914 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560539+0000", 2024-09-16T11:44:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560539+0000", 2024-09-16T11:44:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560539+0000", 2024-09-16T11:44:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:44:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560540+0000", 2024-09-16T11:44:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9899999999999999e-07 2024-09-16T11:44:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560543+0000", 2024-09-16T11:44:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7520000000000001e-06 2024-09-16T11:44:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560545+0000", 2024-09-16T11:44:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9740000000000001e-06 2024-09-16T11:44:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588791+0000", 2024-09-16T11:44:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028245856 2024-09-16T11:44:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588864+0000", 2024-09-16T11:44:01.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3096000000000006e-05 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588932+0000", 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8281000000000006e-05 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20619 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7610368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.2787711389999998, 2024-09-16T11:44:01.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028500152000000001, 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20619 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560560+0000", 2024-09-16T11:44:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9499999999999998e-07 2024-09-16T11:44:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560561+0000", 2024-09-16T11:44:01.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.071e-06 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560565+0000", 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560567+0000", 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588949+0000", 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381878999999999 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588989+0000", 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0649000000000003e-05 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589059+0000", 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0239000000000003e-05 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20620 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7716864~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.2787617390000001, 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028627621999999998, 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20620 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562377+0000", 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018081729999999999 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562380+0000", 2024-09-16T11:44:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0469999999999998e-06 2024-09-16T11:44:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562386+0000", 2024-09-16T11:44:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2040000000000002e-06 2024-09-16T11:44:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562390+0000", 2024-09-16T11:44:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:44:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589092+0000", 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026701798999999998 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589139+0000", 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7320000000000001e-05 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589196+0000", 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7201999999999999e-05 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20626 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 8142848~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.562442+0000", 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.2768883500000001, 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028046026000000002, 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20626 2024-09-16T11:44:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562442+0000", 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562442+0000", 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562442+0000", 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562444+0000", 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0729999999999999e-06 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562446+0000", 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8320000000000002e-06 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562448+0000", 2024-09-16T11:44:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:44:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.590347+0000", 2024-09-16T11:44:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027898284999999998 2024-09-16T11:44:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.590417+0000", 2024-09-16T11:44:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0205999999999998e-05 2024-09-16T11:44:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.590488+0000", 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1297999999999994e-05 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20627 4.5 4.e2161ba5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.562450+0000", 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.2768801960000005, 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028076170000000001, 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-16T11:44:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20627 2024-09-16T11:44:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562450+0000", 2024-09-16T11:44:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562450+0000", 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562450+0000", 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0800000000000001e-07 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562452+0000", 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.06e-06 2024-09-16T11:44:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562454+0000", 2024-09-16T11:44:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.932e-06 2024-09-16T11:44:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562456+0000", 2024-09-16T11:44:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:44:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.590517+0000", 2024-09-16T11:44:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028060259000000001 2024-09-16T11:44:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.590526+0000", 2024-09-16T11:44:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7929999999999992e-06 2024-09-16T11:44:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:01.935 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:01.937 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:01.938 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:01.934+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:01.938 INFO:teuthology.orchestra.run.smithi070.stdout:3235 write 0x5878e0 thru 0x58976d (0x1e8e bytes) 2024-09-16T11:44:01.938 INFO:teuthology.orchestra.run.smithi070.stdout:3236 trunc from 0x4970f78 to 0x27d11ae 2024-09-16T11:44:01.938 INFO:teuthology.orchestra.run.smithi070.stdout:3237 read 0x22c098e thru 0x22c9a0f (0x9082 bytes) 2024-09-16T11:44:01.939 INFO:teuthology.orchestra.run.smithi070.stdout:3238 writesame 0xd091ee0 thru 0xd09e237 data_size 0x5e(0xc358 bytes) 2024-09-16T11:44:01.939 INFO:teuthology.orchestra.run.smithi070.stdout:3239 trunc from 0xd09e238 to 0xe02c2cd 2024-09-16T11:44:01.939 INFO:teuthology.orchestra.run.smithi070.stdout:3241 trunc from 0xe02c2cd to 0x5ff8b3d 2024-09-16T11:44:01.939 INFO:teuthology.orchestra.run.smithi070.stdout:3242 read 0x206ecef thru 0x207241c (0x372e bytes) 2024-09-16T11:44:01.939 INFO:teuthology.orchestra.run.smithi070.stdout:3243 punch from 0x30ed3d3 to 0x30fbb2d, (0xe75a bytes) 2024-09-16T11:44:01.939 INFO:teuthology.orchestra.run.smithi070.stdout:3245 writesame 0x20f849e thru 0x20fa4c5 data_size 0x38(0x2028 bytes) 2024-09-16T11:44:01.939 INFO:teuthology.orchestra.run.smithi070.stdout:3246 read 0x4cc1ca9 thru 0x4cc8024 (0x637c bytes) 2024-09-16T11:44:01.939 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.939 INFO:teuthology.orchestra.run.smithi070.stdout:3250 write 0xb6f1c6f thru 0xb6fbeae (0xa240 bytes) 2024-09-16T11:44:01.939 INFO:teuthology.orchestra.run.smithi070.stdout:3251 write 0xd71e527 thru 0xd726a59 (0x8533 bytes) 2024-09-16T11:44:01.939 INFO:teuthology.orchestra.run.smithi070.stdout:3252 write 0xb5b1e03 thru 0xb5c132c (0xf52a bytes) 2024-09-16T11:44:01.942 INFO:teuthology.orchestra.run.smithi070.stdout:3253 read 0x3fea307 thru 0x3feff73 (0x5c6d bytes) 2024-09-16T11:44:01.944 INFO:teuthology.orchestra.run.smithi070.stdout:3254 trunc from 0xd726a5a to 0xea33680 2024-09-16T11:44:01.949 INFO:teuthology.orchestra.run.smithi070.stdout:3255 read 0xbd9c8a4 thru 0xbda4f5b (0x86b8 bytes) 2024-09-16T11:44:01.949 INFO:teuthology.orchestra.run.smithi070.stdout:3256 write 0x233f389 thru 0x2344d4c (0x59c4 bytes) 2024-09-16T11:44:01.951 INFO:teuthology.orchestra.run.smithi070.stdout:3258 write 0x48e655e thru 0x48efd2b (0x97ce bytes) 2024-09-16T11:44:01.958 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.958 INFO:teuthology.orchestra.run.smithi070.stdout:3260 read 0xacf1086 thru 0xacf26bc (0x1637 bytes) 2024-09-16T11:44:01.958 INFO:teuthology.orchestra.run.smithi070.stdout:3261 read 0x48e0655 thru 0x48e103c (0x9e8 bytes) 2024-09-16T11:44:01.958 INFO:teuthology.orchestra.run.smithi070.stdout:3262 write 0xae0d5db thru 0xae198f2 (0xc318 bytes) 2024-09-16T11:44:01.961 INFO:teuthology.orchestra.run.smithi070.stdout:3264 write 0x960166 thru 0x965072 (0x4f0d bytes) 2024-09-16T11:44:01.962 INFO:teuthology.orchestra.run.smithi070.stdout:3265 read 0x496a21e thru 0x4976770 (0xc553 bytes) 2024-09-16T11:44:01.963 INFO:teuthology.orchestra.run.smithi070.stdout:3266 read 0x3151156 thru 0x3156f52 (0x5dfd bytes) 2024-09-16T11:44:01.963 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:01.975 INFO:teuthology.orchestra.run.smithi070.stdout:3270 trunc from 0xea33680 to 0x44ce76c 2024-09-16T11:44:02.003 INFO:teuthology.orchestra.run.smithi070.stdout:3271 write 0xd334ace thru 0xd3406ba (0xbbed bytes) 2024-09-16T11:44:02.013 INFO:teuthology.orchestra.run.smithi070.stdout:3273 write 0x3f2a276 thru 0x3f2a9e5 (0x770 bytes) 2024-09-16T11:44:02.014 INFO:teuthology.orchestra.run.smithi070.stdout:3274 read 0xaa81083 thru 0xaa8d03b (0xbfb9 bytes) 2024-09-16T11:44:02.014 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.014 INFO:teuthology.orchestra.run.smithi070.stdout:3276 flatten 2024-09-16T11:44:02.021 INFO:teuthology.orchestra.run.smithi070.stdout:3277 read 0x6359359 thru 0x635c965 (0x360d bytes) 2024-09-16T11:44:02.021 INFO:teuthology.orchestra.run.smithi070.stdout:3278 writesame 0x1f04b24 thru 0x1f12306 data_size 0x1(0xd7e3 bytes) 2024-09-16T11:44:02.030 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.030 INFO:teuthology.orchestra.run.smithi070.stdout:3281 writesame 0xadc6961 thru 0xadcb65c data_size 0x1a(0x4cfc bytes) 2024-09-16T11:44:02.034 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.034 INFO:teuthology.orchestra.run.smithi070.stdout:3283 writesame 0xc3f6853 thru 0xc3f903f data_size 0x1(0x27ed bytes) 2024-09-16T11:44:02.036 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:02.035+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:02.038 INFO:teuthology.orchestra.run.smithi070.stdout:3284 read 0xabe4396 thru 0xabeccd3 (0x893e bytes) 2024-09-16T11:44:02.038 INFO:teuthology.orchestra.run.smithi070.stdout:3285 read 0x978810f thru 0x9788a6b (0x95d bytes) 2024-09-16T11:44:02.039 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.039 INFO:teuthology.orchestra.run.smithi070.stdout:3287 read 0x1a1cf0d thru 0x1a1ed87 (0x1e7b bytes) 2024-09-16T11:44:02.039 INFO:teuthology.orchestra.run.smithi070.stdout:3289 read 0x8aac2c8 thru 0x8ab4543 (0x827c bytes) 2024-09-16T11:44:02.039 INFO:teuthology.orchestra.run.smithi070.stdout:3291 writesame 0x4131a33 thru 0x41334cb data_size 0x1(0x1a99 bytes) 2024-09-16T11:44:02.050 INFO:teuthology.orchestra.run.smithi070.stdout:3294 trunc from 0xd3406bb to 0x70f2001 2024-09-16T11:44:02.064 INFO:teuthology.orchestra.run.smithi070.stdout:3295 write 0x40a5771 thru 0x40abe14 (0x66a4 bytes) 2024-09-16T11:44:02.066 INFO:teuthology.orchestra.run.smithi070.stdout:3296 read 0x681a387 thru 0x681cef5 (0x2b6f bytes) 2024-09-16T11:44:02.066 INFO:teuthology.orchestra.run.smithi070.stdout:3297 write 0x49b9170 thru 0x49bb631 (0x24c2 bytes) 2024-09-16T11:44:02.074 INFO:teuthology.orchestra.run.smithi070.stdout:3298 trunc from 0x70f2001 to 0xbe6ef46 2024-09-16T11:44:02.079 INFO:teuthology.orchestra.run.smithi070.stdout:3299 write 0x56f9c49 thru 0x5703ad9 (0x9e91 bytes) 2024-09-16T11:44:02.082 INFO:teuthology.orchestra.run.smithi070.stdout:3300 writesame 0x93bbae7 thru 0x93c9f44 data_size 0x1(0xe45e bytes) 2024-09-16T11:44:02.085 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33426 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'390, mlcod=53'387)", 2024-09-16T11:44:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.083843+0000", 2024-09-16T11:44:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.00031432200000000002, 2024-09-16T11:44:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032841199999999999, 2024-09-16T11:44:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.083843+0000", 2024-09-16T11:44:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.083843+0000", 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.083843+0000", 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.083846+0000", 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.847e-06 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.083852+0000", 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8209999999999998e-06 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.083855+0000", 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6529999999999998e-06 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.083879+0000", 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4769e-05 2024-09-16T11:44:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.083890+0000", 2024-09-16T11:44:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0511e-05 2024-09-16T11:44:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-09-16T11:44:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-09-16T11:44:02.089 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:02.097 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:02.106 INFO:teuthology.orchestra.run.smithi070.stdout:3301 writesame 0xe626fa7 thru 0xe628679 data_size 0x1(0x16d3 bytes) 2024-09-16T11:44:02.106 INFO:teuthology.orchestra.run.smithi070.stdout:3303 read 0x6a99d53 thru 0x6aa459f (0xa84d bytes) 2024-09-16T11:44:02.107 INFO:teuthology.orchestra.run.smithi070.stdout:3305 write 0xd1e9499 thru 0xd1f5a80 (0xc5e8 bytes) 2024-09-16T11:44:02.112 INFO:teuthology.orchestra.run.smithi070.stdout:3306 read 0xca971da thru 0xcaa507d (0xdea4 bytes) 2024-09-16T11:44:02.112 INFO:teuthology.orchestra.run.smithi070.stdout:3308 trunc from 0xe62867a to 0xe91f45b 2024-09-16T11:44:02.117 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.117 INFO:teuthology.orchestra.run.smithi070.stdout:3311 write 0x87ddb1a thru 0x87ed8d9 (0xfdc0 bytes) 2024-09-16T11:44:02.120 INFO:teuthology.orchestra.run.smithi070.stdout:3312 read 0xbeba343 thru 0xbebdd1e (0x39dc bytes) 2024-09-16T11:44:02.120 INFO:teuthology.orchestra.run.smithi070.stdout:3313 read 0xba60baa thru 0xba67f36 (0x738d bytes) 2024-09-16T11:44:02.121 INFO:teuthology.orchestra.run.smithi070.stdout:3315 write 0x523e9fc thru 0x524e1b4 (0xf7b9 bytes) 2024-09-16T11:44:02.122 INFO:teuthology.orchestra.run.smithi070.stdout:3316 read 0x7f9f015 thru 0x7fa9dae (0xad9a bytes) 2024-09-16T11:44:02.122 INFO:teuthology.orchestra.run.smithi070.stdout:3317 read 0x3ee1f4 thru 0x3fb902 (0xd70f bytes) 2024-09-16T11:44:02.125 INFO:teuthology.orchestra.run.smithi070.stdout:3318 trunc from 0xe91f45b to 0xc64e791 2024-09-16T11:44:02.132 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.132 INFO:teuthology.orchestra.run.smithi070.stdout:3321 writesame 0x944b57 thru 0x9518eb data_size 0x35(0xcd95 bytes) 2024-09-16T11:44:02.134 INFO:teuthology.orchestra.run.smithi070.stdout:3322 writesame 0x76eb377 thru 0x76fa1c0 data_size 0x9(0xee4a bytes) 2024-09-16T11:44:02.140 INFO:teuthology.orchestra.run.smithi070.stdout:3324 writesame 0x8b61b14 thru 0x8b66df9 data_size 0x6(0x52e6 bytes) 2024-09-16T11:44:02.141 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:02.139+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:02.142 INFO:teuthology.orchestra.run.smithi070.stdout:3325 read 0x9612ecb thru 0x9616e37 (0x3f6d bytes) 2024-09-16T11:44:02.143 INFO:teuthology.orchestra.run.smithi070.stdout:3326 read 0x619d097 thru 0x61a86e6 (0xb650 bytes) 2024-09-16T11:44:02.143 INFO:teuthology.orchestra.run.smithi070.stdout:3330 read 0xa6a9883 thru 0xa6af4de (0x5c5c bytes) 2024-09-16T11:44:02.143 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.156 INFO:teuthology.orchestra.run.smithi070.stdout:3332 trunc from 0xc64e791 to 0x1441435 2024-09-16T11:44:02.190 INFO:teuthology.orchestra.run.smithi070.stdout:3334 writesame 0x9ad962e thru 0x9adaacc data_size 0x1(0x149f bytes) 2024-09-16T11:44:02.190 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.196 INFO:teuthology.orchestra.run.smithi070.stdout:3338 trunc from 0x9adaacd to 0xe922600 2024-09-16T11:44:02.201 INFO:teuthology.orchestra.run.smithi070.stdout:3339 punch from 0x6d52d36 to 0x6d5e408, (0xb6d2 bytes) 2024-09-16T11:44:02.201 INFO:teuthology.orchestra.run.smithi070.stdout:3340 read 0x31f9531 thru 0x31ff209 (0x5cd9 bytes) 2024-09-16T11:44:02.204 INFO:teuthology.orchestra.run.smithi070.stdout:3341 trunc from 0xe922600 to 0xc1fc2c2 2024-09-16T11:44:02.205 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:02.206 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:02.209 INFO:teuthology.orchestra.run.smithi070.stdout:3344 write 0x7cc5881 thru 0x7cd056c (0xacec bytes) 2024-09-16T11:44:02.215 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:02.221 INFO:teuthology.orchestra.run.smithi070.stdout:3345 punch from 0x6c16929 to 0x6c21821, (0xaef8 bytes) 2024-09-16T11:44:02.221 INFO:teuthology.orchestra.run.smithi070.stdout:3346 writesame 0x2c8d2c4 thru 0x2c992f0 data_size 0x1f(0xc02d bytes) 2024-09-16T11:44:02.235 INFO:teuthology.orchestra.run.smithi070.stdout:3347 read 0xb85d686 thru 0xb8687e5 (0xb160 bytes) 2024-09-16T11:44:02.235 INFO:teuthology.orchestra.run.smithi070.stdout:3348 writesame 0x32d53b6 thru 0x32dffa9 data_size 0x47(0xabf4 bytes) 2024-09-16T11:44:02.238 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:02.237+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:02.240 INFO:teuthology.orchestra.run.smithi070.stdout:3350 read 0xbb8360d thru 0xbb88997 (0x538b bytes) 2024-09-16T11:44:02.240 INFO:teuthology.orchestra.run.smithi070.stdout:3351 punch from 0x61325c9 to 0x61344ee, (0x1f25 bytes) 2024-09-16T11:44:02.240 INFO:teuthology.orchestra.run.smithi070.stdout:3352 punch from 0x2ca178d to 0x2ca7844, (0x60b7 bytes) 2024-09-16T11:44:02.241 INFO:teuthology.orchestra.run.smithi070.stdout:3353 trunc from 0xc1fc2c2 to 0xb942c2a 2024-09-16T11:44:02.253 INFO:teuthology.orchestra.run.smithi070.stdout:3354 write 0xdfe8e0a thru 0xdfe9bee (0xde5 bytes) 2024-09-16T11:44:02.256 INFO:teuthology.orchestra.run.smithi070.stdout:3355 punch from 0x448adfc to 0x4491c60, (0x6e64 bytes) 2024-09-16T11:44:02.256 INFO:teuthology.orchestra.run.smithi070.stdout:3356 write 0x7966146 thru 0x7972551 (0xc40c bytes) 2024-09-16T11:44:02.258 INFO:teuthology.orchestra.run.smithi070.stdout:3357 writesame 0x8a6c04e thru 0x8a78f3f data_size 0x1(0xcef2 bytes) 2024-09-16T11:44:02.272 INFO:teuthology.orchestra.run.smithi070.stdout:3358 trunc from 0xdfe9bef to 0xb013931 2024-09-16T11:44:02.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:44:02.281 INFO:teuthology.orchestra.run.smithi070.stdout:3359 write 0x1776da4 thru 0x1782b33 (0xbd90 bytes) 2024-09-16T11:44:02.283 INFO:teuthology.orchestra.run.smithi070.stdout:3360 trunc from 0xb013931 to 0xb6ec277 2024-09-16T11:44:02.288 INFO:teuthology.orchestra.run.smithi070.stdout:3361 read 0x633e9f6 thru 0x6346701 (0x7d0c bytes) 2024-09-16T11:44:02.289 INFO:teuthology.orchestra.run.smithi070.stdout:3362 write 0x85d3c55 thru 0x85d8f16 (0x52c2 bytes) 2024-09-16T11:44:02.297 INFO:teuthology.orchestra.run.smithi070.stdout:3363 write 0xcd10278 thru 0xcd13323 (0x30ac bytes) 2024-09-16T11:44:02.300 INFO:teuthology.orchestra.run.smithi070.stdout:3364 read 0xaba102e thru 0xaba2045 (0x1018 bytes) 2024-09-16T11:44:02.300 INFO:teuthology.orchestra.run.smithi070.stdout:3365 read 0x2ab8c62 thru 0x2abc52c (0x38cb bytes) 2024-09-16T11:44:02.300 INFO:teuthology.orchestra.run.smithi070.stdout:3367 read 0x49b07a8 thru 0x49bfbad (0xf406 bytes) 2024-09-16T11:44:02.300 INFO:teuthology.orchestra.run.smithi070.stdout:3369 read 0x3d90d51 thru 0x3da012f (0xf3df bytes) 2024-09-16T11:44:02.301 INFO:teuthology.orchestra.run.smithi070.stdout:3370 write 0x7296077 thru 0x7298f67 (0x2ef1 bytes) 2024-09-16T11:44:02.306 INFO:teuthology.orchestra.run.smithi070.stdout:3373 writesame 0x389946 thru 0x391cfa data_size 0x1(0x83b5 bytes) 2024-09-16T11:44:02.316 INFO:teuthology.orchestra.run.smithi070.stdout:3374 read 0x1c27c12 thru 0x1c287f6 (0xbe5 bytes) 2024-09-16T11:44:02.316 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.316 INFO:teuthology.orchestra.run.smithi070.stdout:3376 punch from 0x2586bcb to 0x25941bc, (0xd5f1 bytes) 2024-09-16T11:44:02.316 INFO:teuthology.orchestra.run.smithi070.stdout:3379 write 0x13b0cfe thru 0x13bbe9c (0xb19f bytes) 2024-09-16T11:44:02.330 INFO:teuthology.orchestra.run.smithi070.stdout:3381 trunc from 0xcd13324 to 0x2ec3cdd 2024-09-16T11:44:02.339 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:02.338+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:02.342 INFO:teuthology.orchestra.run.smithi070.stdout:3382 read 0x2118e0a thru 0x2128574 (0xf76b bytes) 2024-09-16T11:44:02.343 INFO:teuthology.orchestra.run.smithi070.stdout:3383 read 0x1438b5f thru 0x1445761 (0xcc03 bytes) 2024-09-16T11:44:02.344 INFO:teuthology.orchestra.run.smithi070.stdout:3385 trunc from 0x2ec3cdd to 0x3984244 2024-09-16T11:44:02.363 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:02.363 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:02.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:02.363 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:02.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.363 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:02.363 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:02.364 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.440553797, 2024-09-16T11:44:02.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:44:02.364 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.364 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:02.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:02.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:02.364 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:44:02.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.364 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:02.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:02.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:02.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:44:02.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:44:02.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:44:02.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:44:02.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:02.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:44:02.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:44:02.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:44:02.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:44:02.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:44:02.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:44:02.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:02.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:44:02.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:44:02.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:02.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:44:02.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:44:02.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:02.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:44:02.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:44:02.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:44:02.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:44:02.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:44:02.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:44:02.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.370 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.370 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:02.370 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:02.370 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.440508104999999, 2024-09-16T11:44:02.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:44:02.370 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.370 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:02.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:02.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:02.371 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:44:02.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.371 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:02.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:02.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:44:02.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:44:02.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:44:02.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:44:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:44:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:44:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:44:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:44:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:44:02.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:44:02.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:44:02.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:44:02.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:02.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:44:02.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:44:02.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:44:02.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:44:02.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.376 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.376 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.440451175, 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:44:02.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:44:02.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:44:02.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:44:02.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:44:02.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:44:02.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:44:02.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:44:02.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:44:02.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:44:02.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:02.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:44:02.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:44:02.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:02.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:44:02.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:44:02.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:02.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:44:02.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:44:02.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:44:02.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:44:02.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:44:02.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:44:02.382 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.382 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.440425806999997, 2024-09-16T11:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:44:02.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.384 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:02.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:02.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:02.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:44:02.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:44:02.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:44:02.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:44:02.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.388 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.440373839000003, 2024-09-16T11:44:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:44:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:02.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:44:02.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:44:02.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:02.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:44:02.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:44:02.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:02.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:44:02.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.440292728999999, 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:44:02.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:44:02.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:44:02.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:44:02.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:44:02.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:02.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:44:02.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:44:02.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:44:02.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:44:02.400 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.400 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.400 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.400 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:02.401 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:02.401 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.440266751000003, 2024-09-16T11:44:02.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:44:02.401 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.401 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.401 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:02.401 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:02.401 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:02.401 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:44:02.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.401 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:02.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:44:02.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:44:02.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:44:02.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:44:02.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:44:02.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.406 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.406 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:02.406 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:02.407 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.022748126000003, 2024-09-16T11:44:02.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:44:02.407 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.407 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:02.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:02.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:02.407 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:44:02.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.407 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:02.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:02.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:44:02.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:44:02.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:44:02.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:44:02.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:44:02.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:44:02.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:44:02.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:44:02.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:44:02.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:44:02.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:44:02.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:44:02.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:44:02.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:44:02.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:44:02.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:44:02.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.412 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.022726194999997, 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:44:02.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:44:02.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:44:02.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:44:02.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:44:02.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:02.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:44:02.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:44:02.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:02.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:44:02.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:44:02.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:02.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:44:02.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:44:02.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:44:02.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:44:02.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:44:02.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:44:02.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.022649307000002, 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:02.419 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:44:02.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.420 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:02.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:02.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:44:02.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:44:02.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:44:02.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:44:02.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:44:02.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:44:02.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:44:02.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:44:02.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:44:02.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:44:02.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28187 4.17 e52/28 4:ee916bd9:::rbd_data.11c1ce740b46.0000000000000000:head v 52'195, mlcod=52'193)", 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.8512299209999998, 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075854429999999999, 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509349+0000", 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2300000000000001e-06 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513315+0000", 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039660590000000001 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513328+0000", 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.308e-05 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513334+0000", 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513385+0000", 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1347000000000002e-05 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513407+0000", 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1852999999999999e-05 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516882+0000", 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034749690000000001 2024-09-16T11:44:02.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516933+0000", 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0952999999999998e-05 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28959 4.8 e53/28 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head v 53'408, mlcod=53'406)", 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.3290808649999999, 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088740420000000004, 2024-09-16T11:44:02.429 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031498+0000", 2024-09-16T11:44:02.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.652e-06 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033515+0000", 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020165930000000001 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033539+0000", 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4295999999999999e-05 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033548+0000", 2024-09-16T11:44:02.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1239999999999994e-06 2024-09-16T11:44:02.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033610+0000", 2024-09-16T11:44:02.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1371999999999994e-05 2024-09-16T11:44:02.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033637+0000", 2024-09-16T11:44:02.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7098999999999999e-05 2024-09-16T11:44:02.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040341+0000", 2024-09-16T11:44:02.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067045819999999997 2024-09-16T11:44:02.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040371+0000", 2024-09-16T11:44:02.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9323999999999999e-05 2024-09-16T11:44:02.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.433 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.434 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32303 4.8 e53/28 4:1436ac82:::rbd_data.11c1cab081b3.0000000000000022:head v 53'417, mlcod=53'415)", 2024-09-16T11:44:02.434 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:02.434 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.85865489100000003, 2024-09-16T11:44:02.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085565160000000001, 2024-09-16T11:44:02.434 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.434 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.434 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:02.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:02.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:02.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-09-16T11:44:02.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501930+0000", 2024-09-16T11:44:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7649999999999996e-06 2024-09-16T11:44:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501946+0000", 2024-09-16T11:44:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5301000000000001e-05 2024-09-16T11:44:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501952+0000", 2024-09-16T11:44:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:44:02.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502100+0000", 2024-09-16T11:44:02.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014847600000000001 2024-09-16T11:44:02.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502120+0000", 2024-09-16T11:44:02.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9303000000000001e-05 2024-09-16T11:44:02.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510436+0000", 2024-09-16T11:44:02.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083166420000000008 2024-09-16T11:44:02.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510479+0000", 2024-09-16T11:44:02.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2979000000000003e-05 2024-09-16T11:44:02.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.438 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.438 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32306 4.1e e53/28 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head v 53'124, mlcod=53'122)", 2024-09-16T11:44:02.438 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.85858889699999996, 2024-09-16T11:44:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084990900000000008, 2024-09-16T11:44:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:02.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:02.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:44:02.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501994+0000", 2024-09-16T11:44:02.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:44:02.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502002+0000", 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3839999999999997e-06 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502007+0000", 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6290000000000001e-06 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502454+0000", 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00044736800000000002 2024-09-16T11:44:02.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502471+0000", 2024-09-16T11:44:02.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6648999999999999e-05 2024-09-16T11:44:02.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510455+0000", 2024-09-16T11:44:02.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0079841909999999999 2024-09-16T11:44:02.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510488+0000", 2024-09-16T11:44:02.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2948999999999999e-05 2024-09-16T11:44:02.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.443 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.443 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32307 4.8 e53/28 4:11c1c37d:::rbd_data.11c1cab081b3.0000000000000026:head v 53'418, mlcod=53'415)", 2024-09-16T11:44:02.443 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:02.443 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.85783356200000005, 2024-09-16T11:44:02.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00776212, 2024-09-16T11:44:02.443 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.443 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:02.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502750+0000", 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502760+0000", 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502767+0000", 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4219999999999997e-06 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502802+0000", 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5089999999999998e-05 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502816+0000", 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4092e-05 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510483+0000", 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00766704 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510506+0000", 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3195999999999999e-05 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33583 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'416, mlcod=53'411)", 2024-09-16T11:44:02.447 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.14987515300000001, 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078132459999999994, 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210703+0000", 2024-09-16T11:44:02.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:02.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210706+0000", 2024-09-16T11:44:02.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-06 2024-09-16T11:44:02.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210712+0000", 2024-09-16T11:44:02.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:44:02.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210715+0000", 2024-09-16T11:44:02.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:44:02.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210736+0000", 2024-09-16T11:44:02.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0293999999999999e-05 2024-09-16T11:44:02.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210746+0000", 2024-09-16T11:44:02.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0178e-05 2024-09-16T11:44:02.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218477+0000", 2024-09-16T11:44:02.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007731501 2024-09-16T11:44:02.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218516+0000", 2024-09-16T11:44:02.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8148000000000001e-05 2024-09-16T11:44:02.452 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.452 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.452 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.452 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33584 4.0 e53/28 4:044e4d16:::rbd_data.11c1cab081b3.0000000000000019:head v 53'176, mlcod=53'174)", 2024-09-16T11:44:02.452 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:02.452 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.14906367000000001, 2024-09-16T11:44:02.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070059470000000002, 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.37e-07 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211524+0000", 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4830000000000005e-06 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211535+0000", 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0842000000000001e-05 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211539+0000", 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9740000000000004e-06 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211584+0000", 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5655000000000002e-05 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211602+0000", 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7756000000000001e-05 2024-09-16T11:44:02.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218497+0000", 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0068944590000000004 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218520+0000", 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3141000000000001e-05 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.456 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33590 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'419, mlcod=53'415)", 2024-09-16T11:44:02.457 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:02.457 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.13881701199999999, 2024-09-16T11:44:02.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082499879999999998, 2024-09-16T11:44:02.457 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.457 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.457 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:02.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221761+0000", 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221765+0000", 2024-09-16T11:44:02.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1590000000000003e-06 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221774+0000", 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7490000000000004e-06 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221778+0000", 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0419999999999996e-06 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221800+0000", 2024-09-16T11:44:02.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1926999999999999e-05 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221810+0000", 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0079e-05 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.229980+0000", 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081698469999999992 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230010+0000", 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0746e-05 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33591 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'420, mlcod=53'415)", 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.13825552799999999, 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007702576, 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:44:02.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222326+0000", 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222337+0000", 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1008e-05 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222342+0000", 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0050000000000004e-06 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222366+0000", 2024-09-16T11:44:02.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3696000000000001e-05 2024-09-16T11:44:02.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222385+0000", 2024-09-16T11:44:02.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-16T11:44:02.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230012+0000", 2024-09-16T11:44:02.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076270940000000001 2024-09-16T11:44:02.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230025+0000", 2024-09-16T11:44:02.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2112e-05 2024-09-16T11:44:02.465 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.465 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.465 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.465 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33593 4.1e e53/28 4:7e77cec2:::rbd_data.11c1cab081b3.000000000000000c:head v 53'133, mlcod=53'131)", 2024-09-16T11:44:02.465 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.133621606, 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070516270000000004, 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:02.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226957+0000", 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227014+0000", 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7074999999999998e-05 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227024+0000", 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0617e-05 2024-09-16T11:44:02.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227030+0000", 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3279999999999996e-06 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227061+0000", 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1718999999999999e-05 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227074+0000", 2024-09-16T11:44:02.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2738000000000001e-05 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.233975+0000", 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0069008480000000002 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.234008+0000", 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2614000000000003e-05 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:02.470 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:02.470 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:02.472 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:02.472 INFO:teuthology.orchestra.run.smithi070.stdout:3386 write 0xeb3e839 thru 0xeb47dff (0x95c7 bytes) 2024-09-16T11:44:02.473 INFO:teuthology.orchestra.run.smithi070.stdout:3387 read 0xae91730 thru 0xae91c06 (0x4d7 bytes) 2024-09-16T11:44:02.473 INFO:teuthology.orchestra.run.smithi070.stdout:3388 read 0x198ca35 thru 0x19966a1 (0x9c6d bytes) 2024-09-16T11:44:02.473 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.473 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.473 INFO:teuthology.orchestra.run.smithi070.stdout:3392 trunc from 0xeb47e00 to 0xa623814 2024-09-16T11:44:02.473 INFO:teuthology.orchestra.run.smithi070.stdout:3395 write 0x5e26e01 thru 0x5e2be04 (0x5004 bytes) 2024-09-16T11:44:02.473 INFO:teuthology.orchestra.run.smithi070.stdout:3396 write 0x79fb62a thru 0x7a0a371 (0xed48 bytes) 2024-09-16T11:44:02.473 INFO:teuthology.orchestra.run.smithi070.stdout:3397 write 0x861b442 thru 0x862b2d3 (0xfe92 bytes) 2024-09-16T11:44:02.473 INFO:teuthology.orchestra.run.smithi070.stdout:3399 write 0x187e6eb thru 0x1888783 (0xa099 bytes) 2024-09-16T11:44:02.473 INFO:teuthology.orchestra.run.smithi070.stdout:3401 write 0x130ba9c thru 0x1312390 (0x68f5 bytes) 2024-09-16T11:44:02.473 INFO:teuthology.orchestra.run.smithi070.stdout:3402 writesame 0x21c6353 thru 0x21d21a4 data_size 0x11(0xbe52 bytes) 2024-09-16T11:44:02.473 INFO:teuthology.orchestra.run.smithi070.stdout:3404 read 0xa17c269 thru 0xa183b76 (0x790e bytes) 2024-09-16T11:44:02.474 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.474 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:02.474 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:02.475 INFO:teuthology.orchestra.run.smithi070.stdout:3407 trunc from 0xa623814 to 0xda85893 2024-09-16T11:44:02.475 INFO:teuthology.orchestra.run.smithi070.stdout:3409 read 0x7abf682 thru 0x7ac4c90 (0x560f bytes) 2024-09-16T11:44:02.475 INFO:teuthology.orchestra.run.smithi070.stdout:3410 read 0x3cfd0e2 thru 0x3d0ab10 (0xda2f bytes) 2024-09-16T11:44:02.475 INFO:teuthology.orchestra.run.smithi070.stdout:3411 writesame 0xe716498 thru 0xe720af1 data_size 0x6b(0xa65a bytes) 2024-09-16T11:44:02.475 INFO:teuthology.orchestra.run.smithi070.stdout:3412 write 0x3dd34c1 thru 0x3ddb16d (0x7cad bytes) 2024-09-16T11:44:02.475 INFO:teuthology.orchestra.run.smithi070.stdout:3413 read 0x9e2d82e thru 0x9e33cf8 (0x64cb bytes) 2024-09-16T11:44:02.475 INFO:teuthology.orchestra.run.smithi070.stdout:3415 write 0xd9e480f thru 0xd9f418f (0xf981 bytes) 2024-09-16T11:44:02.476 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:02.476 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:02.476 INFO:teuthology.orchestra.run.smithi070.stdout:3416 writesame 0x1e4a4be thru 0x1e5173c data_size 0x1(0x727f bytes) 2024-09-16T11:44:02.477 INFO:teuthology.orchestra.run.smithi070.stdout:3417 write 0xdf6345 thru 0xe051ee (0xeeaa bytes) 2024-09-16T11:44:02.477 INFO:teuthology.orchestra.run.smithi070.stdout:3418 writesame 0x940a964 thru 0x941a39d data_size 0x1(0xfa3a bytes) 2024-09-16T11:44:02.477 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:02.477 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:02.478 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:02.478 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:02.480 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:02.470+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:02.481 INFO:teuthology.orchestra.run.smithi070.stdout:3419 trunc from 0xe720af2 to 0x12b0503 2024-09-16T11:44:02.483 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:02.484 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:02.497 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.501 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:02.501 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:02.503 INFO:teuthology.orchestra.run.smithi070.stdout:3421 write 0x261d3df thru 0x26279b7 (0xa5d9 bytes) 2024-09-16T11:44:02.512 INFO:teuthology.orchestra.run.smithi070.stdout:3422 read 0xb2ed39 thru 0xb3cb9a (0xde62 bytes) 2024-09-16T11:44:02.520 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:02.520 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:02.520 INFO:teuthology.orchestra.run.smithi070.stdout:3424 writesame 0x46abc35 thru 0x46af6e4 data_size 0x18(0x3ab0 bytes) 2024-09-16T11:44:02.521 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.533 INFO:teuthology.orchestra.run.smithi070.stdout:3426 write 0xba35332 thru 0xba3a5ce (0x529d bytes) 2024-09-16T11:44:02.537 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:02.537 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:02.537 INFO:teuthology.orchestra.run.smithi070.stdout:3427 write 0x9c6ae31 thru 0x9c6ed3e (0x3f0e bytes) 2024-09-16T11:44:02.548 INFO:teuthology.orchestra.run.smithi070.stdout:3429 writesame 0xc140f4a thru 0xc144054 data_size 0x51(0x310b bytes) 2024-09-16T11:44:02.548 INFO:teuthology.orchestra.run.smithi070.stdout:3430 read 0x37eddec thru 0x37f632f (0x8544 bytes) 2024-09-16T11:44:02.548 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.549 INFO:teuthology.orchestra.run.smithi070.stdout:3433 punch from 0xadc692b to 0xadd522e, (0xe903 bytes) 2024-09-16T11:44:02.550 INFO:teuthology.orchestra.run.smithi070.stdout:3434 trunc from 0xc144055 to 0xd5566dd 2024-09-16T11:44:02.554 INFO:teuthology.orchestra.run.smithi070.stdout:3435 read 0x8ebf10a thru 0x8ec989d (0xa794 bytes) 2024-09-16T11:44:02.555 INFO:teuthology.orchestra.run.smithi070.stdout:3436 read 0x4b3ce21 thru 0x4b48906 (0xbae6 bytes) 2024-09-16T11:44:02.555 INFO:teuthology.orchestra.run.smithi070.stdout:3437 writesame 0xcd5421f thru 0xcd5f7ab data_size 0x1(0xb58d bytes) 2024-09-16T11:44:02.566 INFO:teuthology.orchestra.run.smithi070.stdout:3438 read 0xcffe9d2 thru 0xd00da07 (0xf036 bytes) 2024-09-16T11:44:02.566 INFO:teuthology.orchestra.run.smithi070.stdout:3439 write 0x1142479 thru 0x1143b6c (0x16f4 bytes) 2024-09-16T11:44:02.568 INFO:teuthology.orchestra.run.smithi070.stdout:3440 write 0x75b94f6 thru 0x75ba3c6 (0xed1 bytes) 2024-09-16T11:44:02.571 INFO:teuthology.orchestra.run.smithi070.stdout:3441 punch from 0x946743e to 0x94712f3, (0x9eb5 bytes) 2024-09-16T11:44:02.571 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.571 INFO:teuthology.orchestra.run.smithi070.stdout:3443 read 0x67d6c15 thru 0x67ddb4d (0x6f39 bytes) 2024-09-16T11:44:02.571 INFO:teuthology.orchestra.run.smithi070.stdout:3444 writesame 0xc45a4d thru 0xc49592 data_size 0x1b(0x3b46 bytes) 2024-09-16T11:44:02.571 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:02.570+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:02.572 INFO:teuthology.orchestra.run.smithi070.stdout:3445 writesame 0x3eaee11 thru 0x3eb5b35 data_size 0x1(0x6d25 bytes) 2024-09-16T11:44:02.591 INFO:teuthology.orchestra.run.smithi070.stdout:3447 write 0xdb75207 thru 0xdb75b39 (0x933 bytes) 2024-09-16T11:44:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:3448 punch from 0xc6bde38 to 0xc6bfd6c, (0x1f34 bytes) 2024-09-16T11:44:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:3449 read 0x42dfa80 thru 0x42eb32c (0xb8ad bytes) 2024-09-16T11:44:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:3451 read 0x4a12405 thru 0x4a20b8c (0xe788 bytes) 2024-09-16T11:44:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:3452 writesame 0x30fe65f thru 0x31058b6 data_size 0x4(0x7258 bytes) 2024-09-16T11:44:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:3456 read 0xd4720c thru 0xd55f21 (0xed16 bytes) 2024-09-16T11:44:02.597 INFO:teuthology.orchestra.run.smithi070.stdout:3457 read 0x430eb33 thru 0x431b632 (0xcb00 bytes) 2024-09-16T11:44:02.597 INFO:teuthology.orchestra.run.smithi070.stdout:3458 read 0x4ddfeac thru 0x4de68a4 (0x69f9 bytes) 2024-09-16T11:44:02.597 INFO:teuthology.orchestra.run.smithi070.stdout:3459 punch from 0x7cb33e5 to 0x7cb6281, (0x2e9c bytes) 2024-09-16T11:44:02.597 INFO:teuthology.orchestra.run.smithi070.stdout:3461 punch from 0x9ba2acb to 0x9ba7a65, (0x4f9a bytes) 2024-09-16T11:44:02.598 INFO:teuthology.orchestra.run.smithi070.stdout:3462 writesame 0x4286a23 thru 0x4288dd1 data_size 0x3f(0x23af bytes) 2024-09-16T11:44:02.598 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:02.598 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:02.606 INFO:teuthology.orchestra.run.smithi070.stdout:3463 trunc from 0xdb75b3a to 0x98f8fb5 2024-09-16T11:44:02.609 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:02.621 INFO:teuthology.orchestra.run.smithi070.stdout:3464 trunc from 0x98f8fb5 to 0xd5abba9 2024-09-16T11:44:02.625 INFO:teuthology.orchestra.run.smithi070.stdout:3465 punch from 0x8a20c67 to 0x8a25551, (0x48ea bytes) 2024-09-16T11:44:02.625 INFO:teuthology.orchestra.run.smithi070.stdout:3466 punch from 0xc0a9197 to 0xc0a9572, (0x3db bytes) 2024-09-16T11:44:02.637 INFO:teuthology.orchestra.run.smithi070.stdout:3467 trunc from 0xd5abba9 to 0x37bab31 2024-09-16T11:44:02.647 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.647 INFO:teuthology.orchestra.run.smithi070.stdout:3471 read 0x270237f thru 0x2705929 (0x35ab bytes) 2024-09-16T11:44:02.648 INFO:teuthology.orchestra.run.smithi070.stdout:3472 read 0x2587237 thru 0x25875f3 (0x3bd bytes) 2024-09-16T11:44:02.648 INFO:teuthology.orchestra.run.smithi070.stdout:3473 punch from 0x373686a to 0x373c04f, (0x57e5 bytes) 2024-09-16T11:44:02.648 INFO:teuthology.orchestra.run.smithi070.stdout:3474 punch from 0x2f470c9 to 0x2f4a69d, (0x35d4 bytes) 2024-09-16T11:44:02.650 INFO:teuthology.orchestra.run.smithi070.stdout:3475 trunc from 0x37bab31 to 0x222db44 2024-09-16T11:44:02.656 INFO:teuthology.orchestra.run.smithi070.stdout:3476 read 0x174816f thru 0x1749adc (0x196e bytes) 2024-09-16T11:44:02.665 INFO:teuthology.orchestra.run.smithi070.stdout:3478 write 0x5a522e5 thru 0x5a61a4e (0xf76a bytes) 2024-09-16T11:44:02.671 INFO:teuthology.orchestra.run.smithi070.stdout:3479 write 0x3d2e22 thru 0x3ddea2 (0xb081 bytes) 2024-09-16T11:44:02.671 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:02.669+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:02.688 INFO:teuthology.orchestra.run.smithi070.stdout:3480 write 0xd7a63ca thru 0xd7aa3d5 (0x400c bytes) 2024-09-16T11:44:02.691 INFO:teuthology.orchestra.run.smithi070.stdout:3482 writesame 0x7f6b510 thru 0x7f6c92a data_size 0x1(0x141b bytes) 2024-09-16T11:44:02.695 INFO:teuthology.orchestra.run.smithi070.stdout:3483 punch from 0xf991e5 to 0xf9d591, (0x43ac bytes) 2024-09-16T11:44:02.696 INFO:teuthology.orchestra.run.smithi070.stdout:3484 punch from 0xebe88c to 0xec7278, (0x89ec bytes) 2024-09-16T11:44:02.696 INFO:teuthology.orchestra.run.smithi070.stdout:3485 writesame 0xa1a3ae thru 0xa23132 data_size 0x1(0x8d85 bytes) 2024-09-16T11:44:02.703 INFO:teuthology.orchestra.run.smithi070.stdout:3486 write 0x4493c70 thru 0x4499ae2 (0x5e73 bytes) 2024-09-16T11:44:02.705 INFO:teuthology.orchestra.run.smithi070.stdout:3487 trunc from 0xd7aa3d6 to 0xe022f76 2024-09-16T11:44:02.710 INFO:teuthology.orchestra.run.smithi070.stdout:3488 read 0x652194d thru 0x652a5e2 (0x8c96 bytes) 2024-09-16T11:44:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:3490 punch from 0xcfba05d to 0xcfc4695, (0xa638 bytes) 2024-09-16T11:44:02.711 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.724 INFO:teuthology.orchestra.run.smithi070.stdout:3492 trunc from 0xe022f76 to 0x31acab3 2024-09-16T11:44:02.739 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:02.739 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:02.747 INFO:teuthology.orchestra.run.smithi070.stdout:3493 writesame 0x88379aa thru 0x883aa11 data_size 0x4(0x3068 bytes) 2024-09-16T11:44:02.750 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:02.752 INFO:teuthology.orchestra.run.smithi070.stdout:3494 trunc from 0x883aa12 to 0x3f2235e 2024-09-16T11:44:02.767 INFO:teuthology.orchestra.run.smithi070.stdout:3497 write 0x724175a thru 0x724bd49 (0xa5f0 bytes) 2024-09-16T11:44:02.771 INFO:teuthology.orchestra.run.smithi070.stdout:3498 read 0x40aaa59 thru 0x40b108a (0x6632 bytes) 2024-09-16T11:44:02.771 INFO:teuthology.orchestra.run.smithi070.stdout:3501 read 0x4c8e70b thru 0x4c924e7 (0x3ddd bytes) 2024-09-16T11:44:02.771 INFO:teuthology.orchestra.run.smithi070.stdout:3503 writesame 0x336021a thru 0x3366a35 data_size 0x6(0x681c bytes) 2024-09-16T11:44:02.772 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:02.771+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:02.773 INFO:teuthology.orchestra.run.smithi070.stdout:3506 read 0x23f938b thru 0x240016e (0x6de4 bytes) 2024-09-16T11:44:02.773 INFO:teuthology.orchestra.run.smithi070.stdout:3507 write 0x3bdfbc3 thru 0x3bee45a (0xe898 bytes) 2024-09-16T11:44:02.777 INFO:teuthology.orchestra.run.smithi070.stdout:3508 punch from 0x64d49a0 to 0x64e1833, (0xce93 bytes) 2024-09-16T11:44:02.777 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.777 INFO:teuthology.orchestra.run.smithi070.stdout:3510 punch from 0x2d0d328 to 0x2d1711d, (0x9df5 bytes) 2024-09-16T11:44:02.790 INFO:teuthology.orchestra.run.smithi070.stdout:3511 writesame 0xe51624b thru 0xe51cc1b data_size 0x1(0x69d1 bytes) 2024-09-16T11:44:02.790 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.791 INFO:teuthology.orchestra.run.smithi070.stdout:3516 writesame 0x5411409 thru 0x541b40b data_size 0x17(0xa003 bytes) 2024-09-16T11:44:02.795 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.795 INFO:teuthology.orchestra.run.smithi070.stdout:3518 read 0x2b5c187 thru 0x2b65766 (0x95e0 bytes) 2024-09-16T11:44:02.796 INFO:teuthology.orchestra.run.smithi070.stdout:3520 punch from 0x1bd3dc4 to 0x1bdef06, (0xb142 bytes) 2024-09-16T11:44:02.796 INFO:teuthology.orchestra.run.smithi070.stdout:3521 read 0xb69529f thru 0xb69854a (0x32ac bytes) 2024-09-16T11:44:02.796 INFO:teuthology.orchestra.run.smithi070.stdout:3522 punch from 0x961ae31 to 0x961ed3d, (0x3f0c bytes) 2024-09-16T11:44:02.801 INFO:teuthology.orchestra.run.smithi070.stdout:3524 writesame 0xe7e3da1 thru 0xe7ead86 data_size 0x1(0x6fe6 bytes) 2024-09-16T11:44:02.801 INFO:teuthology.orchestra.run.smithi070.stdout:3525 read 0xa9325a4 thru 0xa94159f (0xeffc bytes) 2024-09-16T11:44:02.801 INFO:teuthology.orchestra.run.smithi070.stdout:3526 write 0x9a956bd thru 0x9a9db65 (0x84a9 bytes) 2024-09-16T11:44:02.809 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:02.809 INFO:teuthology.orchestra.run.smithi070.stdout:3530 read 0x6c3caf6 thru 0x6c4b411 (0xe91c bytes) 2024-09-16T11:44:02.810 INFO:teuthology.orchestra.run.smithi070.stdout:3531 read 0x7403f41 thru 0x7407d99 (0x3e59 bytes) 2024-09-16T11:44:02.825 INFO:teuthology.orchestra.run.smithi070.stdout:3532 trunc from 0xe7ead87 to 0x1d80ee3 2024-09-16T11:44:02.841 INFO:teuthology.orchestra.run.smithi070.stdout:3533 read 0x1d7b839 thru 0x1d7c707 (0xecf bytes) 2024-09-16T11:44:02.843 INFO:teuthology.orchestra.run.smithi070.stdout:3534 trunc from 0x1d80ee3 to 0x38484a9 2024-09-16T11:44:02.848 INFO:teuthology.orchestra.run.smithi070.stdout:3536 read 0x1933cfa thru 0x1935fb2 (0x22b9 bytes) 2024-09-16T11:44:02.849 INFO:teuthology.orchestra.run.smithi070.stdout:3537 read 0xdce081 thru 0xddc586 (0xe506 bytes) 2024-09-16T11:44:02.861 INFO:teuthology.orchestra.run.smithi070.stdout:3538 trunc from 0x38484a9 to 0xd4eca09 2024-09-16T11:44:02.863 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:02.863 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:02.866 INFO:teuthology.orchestra.run.smithi070.stdout:3540 write 0x300d594 thru 0x301abc2 (0xd62f bytes) 2024-09-16T11:44:02.871 INFO:teuthology.orchestra.run.smithi070.stdout:3541 write 0x6ebad37 thru 0x6ec303c (0x8306 bytes) 2024-09-16T11:44:02.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:02.874 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:02.872+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:02.875 INFO:teuthology.orchestra.run.smithi070.stdout:3542 clone 20 order 23 su 65536 sc 6 2024-09-16T11:44:02.921 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:44:02.974 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:02.973+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:02.980 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:02.980 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:02.990 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:03.071 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-16T11:44:03.072 INFO:teuthology.orchestra.run.smithi070.stdout:3543 read 0x88ee5b1 thru 0x88f7b18 (0x9568 bytes) 2024-09-16T11:44:03.072 INFO:teuthology.orchestra.run.smithi070.stdout:3546 trunc from 0xd4eca09 to 0xd9830e1 2024-09-16T11:44:03.076 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:03.073+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:03.081 INFO:teuthology.orchestra.run.smithi070.stdout:3547 read 0x6f09378 thru 0x6f12c6a (0x98f3 bytes) 2024-09-16T11:44:03.095 INFO:teuthology.orchestra.run.smithi070.stdout:3548 trunc from 0xd9830e1 to 0x10eb96b 2024-09-16T11:44:03.168 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:03.168 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:03.176 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:03.174+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:03.182 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:03.277 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:03.275+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:03.278 INFO:teuthology.orchestra.run.smithi070.stdout:3549 writesame 0x45cd0f8 thru 0x45d5909 data_size 0x1(0x8812 bytes) 2024-09-16T11:44:03.298 INFO:teuthology.orchestra.run.smithi070.stdout:3550 writesame 0xecc6a15 thru 0xecd4ab6 data_size 0x1(0xe0a2 bytes) 2024-09-16T11:44:03.303 INFO:teuthology.orchestra.run.smithi070.stdout:3552 trunc from 0xecd4ab7 to 0xadc4c85 2024-09-16T11:44:03.308 INFO:teuthology.orchestra.run.smithi070.stdout:3554 punch from 0x86283f1 to 0x8632a64, (0xa673 bytes) 2024-09-16T11:44:03.308 INFO:teuthology.orchestra.run.smithi070.stdout:3556 write 0x70f8f36 thru 0x7108dd2 (0xfe9d bytes) 2024-09-16T11:44:03.318 INFO:teuthology.orchestra.run.smithi070.stdout:3557 trunc from 0xadc4c85 to 0x5e96c0f 2024-09-16T11:44:03.326 INFO:teuthology.orchestra.run.smithi070.stdout:3559 write 0x55780d9 thru 0x55870b6 (0xefde bytes) 2024-09-16T11:44:03.330 INFO:teuthology.orchestra.run.smithi070.stdout:3562 write 0x14e1156 thru 0x14e5df8 (0x4ca3 bytes) 2024-09-16T11:44:03.331 INFO:teuthology.orchestra.run.smithi070.stdout:3563 read 0x1417d3b thru 0x1422324 (0xa5ea bytes) 2024-09-16T11:44:03.332 INFO:teuthology.orchestra.run.smithi070.stdout:3564 punch from 0xe1637f to 0xe1a39a, (0x401b bytes) 2024-09-16T11:44:03.332 INFO:teuthology.orchestra.run.smithi070.stdout:3565 writesame 0x16bc719 thru 0x16c3274 data_size 0x2(0x6b5c bytes) 2024-09-16T11:44:03.338 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:03.342 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:03.343 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:03.347 INFO:teuthology.orchestra.run.smithi070.stdout:3568 write 0x91a4c78 thru 0x91ac5b9 (0x7942 bytes) 2024-09-16T11:44:03.352 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:03.361 INFO:teuthology.orchestra.run.smithi070.stdout:3569 write 0xe38760a thru 0xe38e96f (0x7366 bytes) 2024-09-16T11:44:03.364 INFO:teuthology.orchestra.run.smithi070.stdout:3571 writesame 0xd8892ef thru 0xd88db7d data_size 0x5(0x488f bytes) 2024-09-16T11:44:03.368 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:03.368 INFO:teuthology.orchestra.run.smithi070.stdout:3574 punch from 0x5475a2d to 0x5479213, (0x37e6 bytes) 2024-09-16T11:44:03.368 INFO:teuthology.orchestra.run.smithi070.stdout:3575 read 0xa195eb8 thru 0xa1a12c0 (0xb409 bytes) 2024-09-16T11:44:03.378 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:03.376+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:03.384 INFO:teuthology.orchestra.run.smithi070.stdout:3577 trunc from 0xe38e970 to 0x6a8590 2024-09-16T11:44:03.402 INFO:teuthology.orchestra.run.smithi070.stdout:3581 write 0x2ad1b83 thru 0x2adc101 (0xa57f bytes) 2024-09-16T11:44:03.419 INFO:teuthology.orchestra.run.smithi070.stdout:3582 write 0xc2105a6 thru 0xc212c46 (0x26a1 bytes) 2024-09-16T11:44:03.422 INFO:teuthology.orchestra.run.smithi070.stdout:3583 punch from 0x4994ced to 0x4996980, (0x1c93 bytes) 2024-09-16T11:44:03.422 INFO:teuthology.orchestra.run.smithi070.stdout:3584 write 0x186bac thru 0x193dac (0xd201 bytes) 2024-09-16T11:44:03.424 INFO:teuthology.orchestra.run.smithi070.stdout:3585 read 0x5c4481d thru 0x5c4deb5 (0x9699 bytes) 2024-09-16T11:44:03.424 INFO:teuthology.orchestra.run.smithi070.stdout:3587 punch from 0x10a2363 to 0x10a8273, (0x5f10 bytes) 2024-09-16T11:44:03.424 INFO:teuthology.orchestra.run.smithi070.stdout:3588 write 0x311838a thru 0x31237aa (0xb421 bytes) 2024-09-16T11:44:03.428 INFO:teuthology.orchestra.run.smithi070.stdout:3589 read 0x19f0d3c thru 0x19ffb0a (0xedcf bytes) 2024-09-16T11:44:03.428 INFO:teuthology.orchestra.run.smithi070.stdout:3590 writesame 0x4625a2a thru 0x46318d1 data_size 0x4(0xbea8 bytes) 2024-09-16T11:44:03.433 INFO:teuthology.orchestra.run.smithi070.stdout:3591 read 0x5f2c77 thru 0x602a86 (0xfe10 bytes) 2024-09-16T11:44:03.434 INFO:teuthology.orchestra.run.smithi070.stdout:3592 write 0x3a0461f thru 0x3a0dd37 (0x9719 bytes) 2024-09-16T11:44:03.437 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:03.437 INFO:teuthology.orchestra.run.smithi070.stdout:3594 writesame 0x683edc9 thru 0x683fe4c data_size 0xe(0x1084 bytes) 2024-09-16T11:44:03.440 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:03.440 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:03.440 INFO:teuthology.orchestra.run.smithi070.stdout:3597 read 0x58c60dd thru 0x58d16c9 (0xb5ed bytes) 2024-09-16T11:44:03.440 INFO:teuthology.orchestra.run.smithi070.stdout:3598 punch from 0x3ac6329 to 0x3ac95a3, (0x327a bytes) 2024-09-16T11:44:03.448 INFO:teuthology.orchestra.run.smithi070.stdout:3599 trunc from 0xc212c47 to 0x54a05dc 2024-09-16T11:44:03.466 INFO:teuthology.orchestra.run.smithi070.stdout:3600 writesame 0x6fe9427 thru 0x6fec7ae data_size 0x1(0x3388 bytes) 2024-09-16T11:44:03.466 INFO:teuthology.orchestra.run.smithi070.stdout:3601 read 0x1214a33 thru 0x121f2b4 (0xa882 bytes) 2024-09-16T11:44:03.466 INFO:teuthology.orchestra.run.smithi070.stdout:3603 read 0x127088d thru 0x1275858 (0x4fcc bytes) 2024-09-16T11:44:03.466 INFO:teuthology.orchestra.run.smithi070.stdout:3605 writesame 0x56687c1 thru 0x566d300 data_size 0x56(0x4b40 bytes) 2024-09-16T11:44:03.476 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:03.477 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:03.479 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:03.476+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:03.481 INFO:teuthology.orchestra.run.smithi070.stdout:3606 writesame 0xd1abd98 thru 0xd1b241e data_size 0xd(0x6687 bytes) 2024-09-16T11:44:03.481 INFO:teuthology.orchestra.run.smithi070.stdout:3607 writesame 0x4f5dbbb thru 0x4f63e5e data_size 0x3b(0x62a4 bytes) 2024-09-16T11:44:03.484 INFO:teuthology.orchestra.run.smithi070.stdout:3608 write 0x156ca2b thru 0x157acc1 (0xe297 bytes) 2024-09-16T11:44:03.486 INFO:teuthology.orchestra.run.smithi070.stdout:3609 write 0x3f25fd3 thru 0x3f2dfeb (0x8019 bytes) 2024-09-16T11:44:03.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:03.487 INFO:teuthology.orchestra.run.smithi070.stdout:3610 writesame 0x828b342 thru 0x828f6f9 data_size 0xb(0x43b8 bytes) 2024-09-16T11:44:03.489 INFO:teuthology.orchestra.run.smithi070.stdout:3612 write 0x82b07e9 thru 0x82b2317 (0x1b2f bytes) 2024-09-16T11:44:03.492 INFO:teuthology.orchestra.run.smithi070.stdout:3613 clone 21 order 21 su 262144 sc 10 2024-09-16T11:44:03.580 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:03.578+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:03.586 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:03.586 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:03.594 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:03.681 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:03.678+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:03.733 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:03.734 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:03.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:03.782 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:03.780+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:03.840 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:03.841 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:03.850 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:03.890 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T11:44:03.890 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:03.888+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:03.941 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone20 from 0xd1b241f (overlap 0x6a8590) to 0x3fe9bc 2024-09-16T11:44:03.958 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:03.959 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:03.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:03.984 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:03.982+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:04.005 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-16T11:44:04.085 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:04.083+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:04.116 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:04.116 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:04.125 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:04.186 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:04.184+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:04.222 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:04.222 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:04.231 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:04.249 INFO:teuthology.orchestra.run.smithi070.stdout:3614 write 0xb0afa14 thru 0xb0bcf7a (0xd567 bytes) 2024-09-16T11:44:04.265 INFO:teuthology.orchestra.run.smithi070.stdout:3615 trunc from 0xd1b241f to 0x62c3b33 2024-09-16T11:44:04.287 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:04.285+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:04.291 INFO:teuthology.orchestra.run.smithi070.stdout:3616 read 0x296c3c8 thru 0x29710c8 (0x4d01 bytes) 2024-09-16T11:44:04.291 INFO:teuthology.orchestra.run.smithi070.stdout:3617 read 0xe5641c thru 0xe5c109 (0x5cee bytes) 2024-09-16T11:44:04.309 INFO:teuthology.orchestra.run.smithi070.stdout:3618 write 0xe42e37e thru 0xe4391f3 (0xae76 bytes) 2024-09-16T11:44:04.312 INFO:teuthology.orchestra.run.smithi070.stdout:3619 write 0x290ea2c thru 0x291b6f4 (0xccc9 bytes) 2024-09-16T11:44:04.326 INFO:teuthology.orchestra.run.smithi070.stdout:3621 trunc from 0xe4391f4 to 0x5dca534 2024-09-16T11:44:04.336 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:04.336 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:04.345 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:04.357 INFO:teuthology.orchestra.run.smithi070.stdout:3622 writesame 0x8f27ea3 thru 0x8f35504 data_size 0x9(0xd662 bytes) 2024-09-16T11:44:04.361 INFO:teuthology.orchestra.run.smithi070.stdout:3623 trunc from 0x8f35505 to 0x6476e3b 2024-09-16T11:44:04.365 INFO:teuthology.orchestra.run.smithi070.stdout:3624 read 0x28d0354 thru 0x28deb86 (0xe833 bytes) 2024-09-16T11:44:04.366 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:04.366 INFO:teuthology.orchestra.run.smithi070.stdout:3626 punch from 0x15164d7 to 0x15174f2, (0x101b bytes) 2024-09-16T11:44:04.378 INFO:teuthology.orchestra.run.smithi070.stdout:3627 write 0xbd9c617 thru 0xbda8dd3 (0xc7bd bytes) 2024-09-16T11:44:04.381 INFO:teuthology.orchestra.run.smithi070.stdout:3628 read 0x5775bd8 thru 0x578189a (0xbcc3 bytes) 2024-09-16T11:44:04.381 INFO:teuthology.orchestra.run.smithi070.stdout:3630 read 0x5b83c2c thru 0x5b8aa80 (0x6e55 bytes) 2024-09-16T11:44:04.381 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:04.382 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:04.382 INFO:teuthology.orchestra.run.smithi070.stdout:3635 trunc from 0xbda8dd4 to 0xbc88615 2024-09-16T11:44:04.388 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:04.385+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:04.389 INFO:teuthology.orchestra.run.smithi070.stdout:3636 trunc from 0xbc88615 to 0xa5b908f 2024-09-16T11:44:04.398 INFO:teuthology.orchestra.run.smithi070.stdout:3637 write 0x823425f thru 0x823bc57 (0x79f9 bytes) 2024-09-16T11:44:04.401 INFO:teuthology.orchestra.run.smithi070.stdout:3638 read 0xa4f4c20 thru 0xa5034ef (0xe8d0 bytes) 2024-09-16T11:44:04.401 INFO:teuthology.orchestra.run.smithi070.stdout:3640 writesame 0x1f590ac thru 0x1f62e34 data_size 0x3(0x9d89 bytes) 2024-09-16T11:44:04.411 INFO:teuthology.orchestra.run.smithi070.stdout:3642 read 0x1900702 thru 0x190c056 (0xb955 bytes) 2024-09-16T11:44:04.414 INFO:teuthology.orchestra.run.smithi070.stdout:3644 trunc from 0xa5b908f to 0x7b84a42 2024-09-16T11:44:04.421 INFO:teuthology.orchestra.run.smithi070.stdout:3647 read 0x21b0c70 thru 0x21bff7a (0xf30b bytes) 2024-09-16T11:44:04.431 INFO:teuthology.orchestra.run.smithi070.stdout:3648 write 0xb638f90 thru 0xb6416c3 (0x8734 bytes) 2024-09-16T11:44:04.435 INFO:teuthology.orchestra.run.smithi070.stdout:3649 punch from 0xb607fd0 to 0xb60b664, (0x3694 bytes) 2024-09-16T11:44:04.435 INFO:teuthology.orchestra.run.smithi070.stdout:3650 read 0x110079 thru 0x11e97c (0xe904 bytes) 2024-09-16T11:44:04.435 INFO:teuthology.orchestra.run.smithi070.stdout:3652 clone 22 order 22 su 65536 sc 5 2024-09-16T11:44:04.485 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:04.485 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:04.489 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:04.487+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:04.494 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:04.553 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:44:04.591 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:04.591 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:04.598 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:04.596+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:04.600 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:04.667 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:04.668 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:04.682 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:04.683 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:04.689 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:04.687+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:04.697 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:04.697 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:04.699 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:04.699 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:04.707 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:04.712 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:04.712 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:04.727 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:04.728 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:04.743 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:04.743 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:04.758 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:04.773 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:04.773 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:04.824 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:04.822+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:04.846 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:04.847 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:04.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:04.847 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:04.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.847 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19942 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1990656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.847 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:04.847 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3177736329999998, 2024-09-16T11:44:04.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027866354999999999, 2024-09-16T11:44:04.847 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.847 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.848 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19942 2024-09-16T11:44:04.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.848 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:04.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:04.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4899999999999995e-07 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526281+0000", 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526287+0000", 2024-09-16T11:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2199999999999997e-06 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526291+0000", 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554068+0000", 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027777076000000001 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554104+0000", 2024-09-16T11:44:04.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5537000000000002e-05 2024-09-16T11:44:04.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554145+0000", 2024-09-16T11:44:04.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1350000000000002e-05 2024-09-16T11:44:04.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.851 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.851 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19943 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2416640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.851 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3177573299999992, 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027947257999999999, 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19943 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:04.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:04.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:04.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5999999999999999e-07 2024-09-16T11:44:04.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526297+0000", 2024-09-16T11:44:04.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8059999999999999e-06 2024-09-16T11:44:04.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526303+0000", 2024-09-16T11:44:04.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6579999999999999e-06 2024-09-16T11:44:04.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526306+0000", 2024-09-16T11:44:04.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3969999999999998e-06 2024-09-16T11:44:04.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554165+0000", 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027858519000000002 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554201+0000", 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5908000000000002e-05 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554242+0000", 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1510000000000001e-05 2024-09-16T11:44:04.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19944 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2842624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3177424349999995, 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028052639000000001, 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.856 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19944 2024-09-16T11:44:04.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.857 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:04.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:04.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:04.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:44:04.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526312+0000", 2024-09-16T11:44:04.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:44:04.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526317+0000", 2024-09-16T11:44:04.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5910000000000003e-06 2024-09-16T11:44:04.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526321+0000", 2024-09-16T11:44:04.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2339999999999999e-06 2024-09-16T11:44:04.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554263+0000", 2024-09-16T11:44:04.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027942181 2024-09-16T11:44:04.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554307+0000", 2024-09-16T11:44:04.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4499999999999997e-05 2024-09-16T11:44:04.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554362+0000", 2024-09-16T11:44:04.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5078999999999999e-05 2024-09-16T11:44:04.860 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.860 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.860 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19945 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3162112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3177278799999996, 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028145673999999999, 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19945 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526325+0000", 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526326+0000", 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.733e-06 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526332+0000", 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526336+0000", 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8410000000000002e-06 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554389+0000", 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028053576 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554427+0000", 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7299999999999999e-05 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554470+0000", 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3195000000000002e-05 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.864 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19946 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3268608~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3177123010000003, 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028232587, 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19946 2024-09-16T11:44:04.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:04.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:04.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526346+0000", 2024-09-16T11:44:04.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6459999999999998e-06 2024-09-16T11:44:04.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526352+0000", 2024-09-16T11:44:04.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1070000000000001e-06 2024-09-16T11:44:04.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526355+0000", 2024-09-16T11:44:04.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:44:04.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554493+0000", 2024-09-16T11:44:04.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.02813746 2024-09-16T11:44:04.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554530+0000", 2024-09-16T11:44:04.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6881999999999997e-05 2024-09-16T11:44:04.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554572+0000", 2024-09-16T11:44:04.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2688999999999997e-05 2024-09-16T11:44:04.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.869 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.869 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19947 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4014080~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.869 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:04.869 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3176929469999994, 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028315646999999999, 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19947 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526360+0000", 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.01e-07 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526361+0000", 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8300000000000001e-06 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526369+0000", 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526377+0000", 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0909999999999993e-06 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554593+0000", 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028216657999999999 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554632+0000", 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8828000000000002e-05 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554675+0000", 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2710000000000003e-05 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.873 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19948 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3588096~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3176705480000006, 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028394545, 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19948 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:04.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:04.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9200000000000001e-07 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526384+0000", 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526391+0000", 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5150000000000003e-06 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526394+0000", 2024-09-16T11:44:04.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4240000000000002e-06 2024-09-16T11:44:04.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554695+0000", 2024-09-16T11:44:04.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028301218999999999 2024-09-16T11:44:04.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554731+0000", 2024-09-16T11:44:04.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6298999999999999e-05 2024-09-16T11:44:04.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554776+0000", 2024-09-16T11:44:04.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4675999999999997e-05 2024-09-16T11:44:04.878 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.878 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.878 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.878 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19949 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1351680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.878 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:04.878 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3176542760000007, 2024-09-16T11:44:04.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028490081, 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19949 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:04.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:04.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:44:04.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526400+0000", 2024-09-16T11:44:04.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7099999999999999e-06 2024-09-16T11:44:04.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526406+0000", 2024-09-16T11:44:04.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5140000000000001e-06 2024-09-16T11:44:04.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526409+0000", 2024-09-16T11:44:04.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5360000000000001e-06 2024-09-16T11:44:04.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554801+0000", 2024-09-16T11:44:04.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028392378999999999 2024-09-16T11:44:04.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554843+0000", 2024-09-16T11:44:04.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1056000000000001e-05 2024-09-16T11:44:04.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554888+0000", 2024-09-16T11:44:04.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466999999999997e-05 2024-09-16T11:44:04.882 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19950 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1777664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3176392799999999, 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028661844999999998, 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.883 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19950 2024-09-16T11:44:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526415+0000", 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.776e-06 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526422+0000", 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6930000000000001e-06 2024-09-16T11:44:04.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526425+0000", 2024-09-16T11:44:04.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:04.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554923+0000", 2024-09-16T11:44:04.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028498102000000001 2024-09-16T11:44:04.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555000+0000", 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6619999999999995e-05 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555075+0000", 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4931999999999993e-05 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19951 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2097152~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3176234690000008, 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028755718, 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19951 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526431+0000", 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:44:04.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526441+0000", 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7790000000000007e-06 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526445+0000", 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.634e-06 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555100+0000", 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654722000000001 2024-09-16T11:44:04.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555138+0000", 2024-09-16T11:44:04.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7996e-05 2024-09-16T11:44:04.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555184+0000", 2024-09-16T11:44:04.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6427999999999997e-05 2024-09-16T11:44:04.891 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.891 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.891 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.892 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19952 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2203648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.892 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:04.892 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3176029089999997, 2024-09-16T11:44:04.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028901157, 2024-09-16T11:44:04.892 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.892 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.892 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.892 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.892 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.892 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19952 2024-09-16T11:44:04.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.892 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526450+0000", 2024-09-16T11:44:04.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:44:04.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526452+0000", 2024-09-16T11:44:04.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9039999999999999e-06 2024-09-16T11:44:04.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526465+0000", 2024-09-16T11:44:04.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3559e-05 2024-09-16T11:44:04.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526470+0000", 2024-09-16T11:44:04.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-09-16T11:44:04.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555217+0000", 2024-09-16T11:44:04.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028747287999999999 2024-09-16T11:44:04.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555272+0000", 2024-09-16T11:44:04.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4885000000000001e-05 2024-09-16T11:44:04.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555350+0000", 2024-09-16T11:44:04.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8428000000000002e-05 2024-09-16T11:44:04.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.896 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.896 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19953 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2523136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.896 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:04.896 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3175783170000006, 2024-09-16T11:44:04.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029004309999999998, 2024-09-16T11:44:04.897 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.897 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.897 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19953 2024-09-16T11:44:04.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.897 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:04.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526476+0000", 2024-09-16T11:44:04.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9810000000000002e-06 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526482+0000", 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526485+0000", 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2760000000000001e-06 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555383+0000", 2024-09-16T11:44:04.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028898113 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555433+0000", 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9753999999999998e-05 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555478+0000", 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5006999999999998e-05 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19954 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2949120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3175633199999996, 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029165054999999999, 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19954 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1799999999999995e-07 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526491+0000", 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7320000000000001e-06 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526497+0000", 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6869999999999999e-06 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526500+0000", 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:44:04.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555513+0000", 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029012626 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555576+0000", 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3288999999999997e-05 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555654+0000", 2024-09-16T11:44:04.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7861999999999994e-05 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19955 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3375104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3175440179999995, 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029241979000000001, 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19955 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526509+0000", 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526510+0000", 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.844e-06 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526516+0000", 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9039999999999997e-06 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526520+0000", 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3639999999999999e-06 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555669+0000", 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029149628 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555706+0000", 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7017000000000002e-05 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555750+0000", 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3856999999999997e-05 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19956 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3694592~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3175290630000003, 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029398799999999999, 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19956 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526524+0000", 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526525+0000", 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526531+0000", 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6010000000000002e-06 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526535+0000", 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.597e-06 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555785+0000", 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029250235999999999 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555845+0000", 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0393000000000003e-05 2024-09-16T11:44:04.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555922+0000", 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6710000000000002e-05 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19957 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4120576~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.3175139100000006, 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029481157000000001, 2024-09-16T11:44:04.913 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.914 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.914 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19957 2024-09-16T11:44:04.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.914 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:04.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:04.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526539+0000", 2024-09-16T11:44:04.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:04.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526540+0000", 2024-09-16T11:44:04.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.759e-06 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526546+0000", 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1079999999999998e-06 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526550+0000", 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.269e-06 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555939+0000", 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029389444000000001 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555977+0000", 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7487999999999998e-05 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.556019+0000", 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2756999999999997e-05 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.917 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.918 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:28959 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:04.918 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:04.918 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.8346574979999999, 2024-09-16T11:44:04.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031145285000000002, 2024-09-16T11:44:04.918 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.918 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:04.918 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.918 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.918 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.918 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 28959 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009396+0000", 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.296e-06 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030864+0000", 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.021468451 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030892+0000", 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7555999999999998e-05 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030902+0000", 2024-09-16T11:44:04.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9550000000000004e-06 2024-09-16T11:44:04.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030979+0000", 2024-09-16T11:44:04.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7137000000000004e-05 2024-09-16T11:44:04.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031051+0000", 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1475000000000003e-05 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031389+0000", 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033827100000000001 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.039264+0000", 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078751700000000008 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040484+0000", 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012199579999999999 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040517+0000", 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2960999999999998e-05 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040540+0000", 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3054999999999999e-05 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.923 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30605 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 3915776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.6003512390000001, 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938655999999999, 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30605 2024-09-16T11:44:04.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.925 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:04.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:04.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:04.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:04.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243704+0000", 2024-09-16T11:44:04.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2199999999999999e-06 2024-09-16T11:44:04.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243710+0000", 2024-09-16T11:44:04.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6340000000000003e-06 2024-09-16T11:44:04.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243714+0000", 2024-09-16T11:44:04.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5030000000000002e-06 2024-09-16T11:44:04.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271450+0000", 2024-09-16T11:44:04.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027736035999999999 2024-09-16T11:44:04.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271536+0000", 2024-09-16T11:44:04.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6521999999999996e-05 2024-09-16T11:44:04.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271640+0000", 2024-09-16T11:44:04.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103341 2024-09-16T11:44:04.928 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.928 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.928 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30607 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4046848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.600328084, 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028130446, 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30607 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.929 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:04.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:04.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243725+0000", 2024-09-16T11:44:04.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2399999999999998e-07 2024-09-16T11:44:04.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243727+0000", 2024-09-16T11:44:04.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3379999999999999e-06 2024-09-16T11:44:04.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243733+0000", 2024-09-16T11:44:04.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4280000000000001e-06 2024-09-16T11:44:04.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243737+0000", 2024-09-16T11:44:04.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8990000000000003e-06 2024-09-16T11:44:04.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271676+0000", 2024-09-16T11:44:04.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938481000000001 2024-09-16T11:44:04.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271757+0000", 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1396999999999993e-05 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271855+0000", 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7379000000000004e-05 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30608 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:04.933 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:04.934 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.6003112330000002, 2024-09-16T11:44:04.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028323879, 2024-09-16T11:44:04.934 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:04.934 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:04.934 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:04.934 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:04.934 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:04.934 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30608 2024-09-16T11:44:04.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.934 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:04.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:04.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:04.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:04.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:04.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:04.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:04.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:04.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:44:04.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:04.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243743+0000", 2024-09-16T11:44:04.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.762e-06 2024-09-16T11:44:04.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:04.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243749+0000", 2024-09-16T11:44:04.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5670000000000002e-06 2024-09-16T11:44:04.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:04.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243758+0000", 2024-09-16T11:44:04.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8309999999999996e-06 2024-09-16T11:44:04.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:04.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271879+0000", 2024-09-16T11:44:04.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028120501999999999 2024-09-16T11:44:04.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:04.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271965+0000", 2024-09-16T11:44:04.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5755999999999997e-05 2024-09-16T11:44:04.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:04.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:04.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:04.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.272065+0000", 2024-09-16T11:44:04.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000100101 2024-09-16T11:44:04.938 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.938 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.938 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.938 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:04.938 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:04.938 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:04.940 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:04.940 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:04.937+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:04.953 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone21 from 0xb6416c4 (overlap 0x5dca534) to 0x384631f 2024-09-16T11:44:05.040 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:05.038+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:05.074 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-16T11:44:05.085 INFO:teuthology.orchestra.run.smithi070.stdout:3655 read 0x3a9be7c thru 0x3a9df2c (0x20b1 bytes) 2024-09-16T11:44:05.086 INFO:teuthology.orchestra.run.smithi070.stdout:3657 write 0x3664b57 thru 0x366f2ab (0xa755 bytes) 2024-09-16T11:44:05.088 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:05.088 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:05.088 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:05.088 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:05.097 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:05.100 INFO:teuthology.orchestra.run.smithi070.stdout:3658 read 0x11908ca thru 0x1191cbc (0x13f3 bytes) 2024-09-16T11:44:05.101 INFO:teuthology.orchestra.run.smithi070.stdout:3660 writesame 0x94c63ad thru 0x94ca04a data_size 0x1(0x3c9e bytes) 2024-09-16T11:44:05.116 INFO:teuthology.orchestra.run.smithi070.stdout:3661 trunc from 0xb6416c4 to 0x678e122 2024-09-16T11:44:05.135 INFO:teuthology.orchestra.run.smithi070.stdout:3662 trunc from 0x678e122 to 0x19f1950 2024-09-16T11:44:05.161 INFO:teuthology.orchestra.run.smithi070.stdout:3663 read 0x11ca930 thru 0x11d5398 (0xaa69 bytes) 2024-09-16T11:44:05.174 INFO:teuthology.orchestra.run.smithi070.stdout:3664 writesame 0x925f446 thru 0x9264046 data_size 0x1(0x4c01 bytes) 2024-09-16T11:44:05.175 INFO:teuthology.orchestra.run.smithi070.stdout:3666 read 0x5ffd296 thru 0x600aba3 (0xd90e bytes) 2024-09-16T11:44:05.181 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:05.179+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:05.183 INFO:teuthology.orchestra.run.smithi070.stdout:3667 write 0xbcf852d thru 0xbd0322e (0xad02 bytes) 2024-09-16T11:44:05.187 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:05.187 INFO:teuthology.orchestra.run.smithi070.stdout:3671 write 0x5f758ec thru 0x5f8443f (0xeb54 bytes) 2024-09-16T11:44:05.191 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:05.191 INFO:teuthology.orchestra.run.smithi070.stdout:3674 writesame 0x50f9967 thru 0x5108ac6 data_size 0x2(0xf160 bytes) 2024-09-16T11:44:05.209 INFO:teuthology.orchestra.run.smithi070.stdout:3676 trunc from 0xbd0322f to 0x7154504 2024-09-16T11:44:05.220 INFO:teuthology.orchestra.run.smithi070.stdout:3677 punch from 0x468e33b to 0x468f0de, (0xda3 bytes) 2024-09-16T11:44:05.220 INFO:teuthology.orchestra.run.smithi070.stdout:3678 punch from 0x58998a1 to 0x589c394, (0x2af3 bytes) 2024-09-16T11:44:05.220 INFO:teuthology.orchestra.run.smithi070.stdout:3679 read 0x6d00fc6 thru 0x6d0dc02 (0xcc3d bytes) 2024-09-16T11:44:05.220 INFO:teuthology.orchestra.run.smithi070.stdout:3680 read 0x1691e98 thru 0x16a1531 (0xf69a bytes) 2024-09-16T11:44:05.221 INFO:teuthology.orchestra.run.smithi070.stdout:3681 trunc from 0x7154504 to 0x5e0c0da 2024-09-16T11:44:05.228 INFO:teuthology.orchestra.run.smithi070.stdout:3682 read 0xa99641 thru 0xaa595b (0xc31b bytes) 2024-09-16T11:44:05.228 INFO:teuthology.orchestra.run.smithi070.stdout:3683 read 0x273a0dc thru 0x27433ff (0x9324 bytes) 2024-09-16T11:44:05.228 INFO:teuthology.orchestra.run.smithi070.stdout:3684 read 0x3830660 thru 0x383e48f (0xde30 bytes) 2024-09-16T11:44:05.234 INFO:teuthology.orchestra.run.smithi070.stdout:3687 trunc from 0x5e0c0da to 0x924db0 2024-09-16T11:44:05.241 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:05.239+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:05.249 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:05.249 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:05.250 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:05.250 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:05.250 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:05.263 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:05.269 INFO:teuthology.orchestra.run.smithi070.stdout:3690 read 0x1698bc thru 0x16c7ab (0x2ef0 bytes) 2024-09-16T11:44:05.274 INFO:teuthology.orchestra.run.smithi070.stdout:3691 write 0xf2e963 thru 0xf313da (0x2a78 bytes) 2024-09-16T11:44:05.275 INFO:teuthology.orchestra.run.smithi070.stdout:3692 read 0xe706fb thru 0xe767bc (0x60c2 bytes) 2024-09-16T11:44:05.290 INFO:teuthology.orchestra.run.smithi070.stdout:3693 write 0x9b179c5 thru 0x9b20c81 (0x92bd bytes) 2024-09-16T11:44:05.304 INFO:teuthology.orchestra.run.smithi070.stdout:3694 trunc from 0x9b20c82 to 0x162e006 2024-09-16T11:44:05.312 INFO:teuthology.orchestra.run.smithi070.stdout:3697 punch from 0x61befc to 0x620d0b, (0x4e0f bytes) 2024-09-16T11:44:05.312 INFO:teuthology.orchestra.run.smithi070.stdout:3698 read 0xd5f9f6 thru 0xd657d9 (0x5de4 bytes) 2024-09-16T11:44:05.334 INFO:teuthology.orchestra.run.smithi070.stdout:3699 write 0xeb330d6 thru 0xeb37f70 (0x4e9b bytes) 2024-09-16T11:44:05.336 INFO:teuthology.orchestra.run.smithi070.stdout:3701 write 0x2485130 thru 0x248e6a8 (0x9579 bytes) 2024-09-16T11:44:05.340 INFO:teuthology.orchestra.run.smithi070.stdout:3702 punch from 0x66d432e to 0x66dc5b5, (0x8287 bytes) 2024-09-16T11:44:05.340 INFO:teuthology.orchestra.run.smithi070.stdout:3703 read 0x64d7e97 thru 0x64e3af8 (0xbc62 bytes) 2024-09-16T11:44:05.340 INFO:teuthology.orchestra.run.smithi070.stdout:3704 read 0xbd3b198 thru 0xbd4ab01 (0xf96a bytes) 2024-09-16T11:44:05.340 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:05.340 INFO:teuthology.orchestra.run.smithi070.stdout:3706 trunc from 0xeb37f71 to 0xe79ba98 2024-09-16T11:44:05.342 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:05.340+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:05.346 INFO:teuthology.orchestra.run.smithi070.stdout:3707 read 0x2d01f8d thru 0x2d0c155 (0xa1c9 bytes) 2024-09-16T11:44:05.346 INFO:teuthology.orchestra.run.smithi070.stdout:3708 write 0x8dae7c8 thru 0x8db8b17 (0xa350 bytes) 2024-09-16T11:44:05.349 INFO:teuthology.orchestra.run.smithi070.stdout:3709 writesame 0x37606a2 thru 0x376b545 data_size 0x2(0xaea4 bytes) 2024-09-16T11:44:05.361 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:05.361 INFO:teuthology.orchestra.run.smithi070.stdout:3712 read 0xae3e108 thru 0xae4679e (0x8697 bytes) 2024-09-16T11:44:05.368 INFO:teuthology.orchestra.run.smithi070.stdout:3713 trunc from 0xe79ba98 to 0x84e8398 2024-09-16T11:44:05.377 INFO:teuthology.orchestra.run.smithi070.stdout:3714 punch from 0x62b790f to 0x62c26b9, (0xadaa bytes) 2024-09-16T11:44:05.377 INFO:teuthology.orchestra.run.smithi070.stdout:3715 writesame 0x6b40f44 thru 0x6b439b6 data_size 0x1(0x2a73 bytes) 2024-09-16T11:44:05.384 INFO:teuthology.orchestra.run.smithi070.stdout:3716 read 0x6da7cba thru 0x6db5834 (0xdb7b bytes) 2024-09-16T11:44:05.384 INFO:teuthology.orchestra.run.smithi070.stdout:3717 write 0x45abd2c thru 0x45afeb7 (0x418c bytes) 2024-09-16T11:44:05.395 INFO:teuthology.orchestra.run.smithi070.stdout:3718 trunc from 0x84e8398 to 0xf583f1 2024-09-16T11:44:05.408 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:9179 4.5 4:a03f235d:::rbd_data.11c1dc795e15.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:44:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:46.201943+0000", 2024-09-16T11:44:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.203954852999999, 2024-09-16T11:44:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028095163999999999, 2024-09-16T11:44:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 9179 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201943+0000", 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201943+0000", 2024-09-16T11:44:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201944+0000", 2024-09-16T11:44:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.283e-06 2024-09-16T11:44:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201948+0000", 2024-09-16T11:44:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0210000000000001e-06 2024-09-16T11:44:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201960+0000", 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1420000000000001e-05 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201965+0000", 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6570000000000002e-06 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.201998+0000", 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2882000000000001e-05 2024-09-16T11:44:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.202046+0000", 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7274000000000002e-05 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.206571+0000", 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045255080000000001 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.229964+0000", 2024-09-16T11:44:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023392901000000001 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.229973+0000", 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8589999999999994e-06 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.230010+0000", 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7261000000000001e-05 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:46.230038+0000", 2024-09-16T11:44:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8098000000000001e-05 2024-09-16T11:44:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19271 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7766016~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472355+0000", 2024-09-16T11:44:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.93354274, 2024-09-16T11:44:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028037484000000001, 2024-09-16T11:44:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19271 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472355+0000", 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472355+0000", 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472356+0000", 2024-09-16T11:44:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:44:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472357+0000", 2024-09-16T11:44:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7010000000000001e-06 2024-09-16T11:44:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472363+0000", 2024-09-16T11:44:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5210000000000002e-06 2024-09-16T11:44:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472366+0000", 2024-09-16T11:44:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.444e-06 2024-09-16T11:44:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500244+0000", 2024-09-16T11:44:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027878028999999999 2024-09-16T11:44:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500320+0000", 2024-09-16T11:44:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5443000000000003e-05 2024-09-16T11:44:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500393+0000", 2024-09-16T11:44:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2849999999999995e-05 2024-09-16T11:44:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19272 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 8192000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472377+0000", 2024-09-16T11:44:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9335208129999994, 2024-09-16T11:44:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028126168999999999, 2024-09-16T11:44:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19272 2024-09-16T11:44:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472377+0000", 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472377+0000", 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472378+0000", 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.84e-07 2024-09-16T11:44:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472379+0000", 2024-09-16T11:44:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8449999999999999e-06 2024-09-16T11:44:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472386+0000", 2024-09-16T11:44:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5549999999999999e-06 2024-09-16T11:44:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472390+0000", 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500422+0000", 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028032715 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500468+0000", 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5411000000000003e-05 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500503+0000", 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-05 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19274 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 5529600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472427+0000", 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9334708549999995, 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028248603000000001, 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19274 2024-09-16T11:44:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472427+0000", 2024-09-16T11:44:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472427+0000", 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472428+0000", 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4600000000000005e-07 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472429+0000", 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472436+0000", 2024-09-16T11:44:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1340000000000001e-06 2024-09-16T11:44:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472444+0000", 2024-09-16T11:44:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9299999999999992e-06 2024-09-16T11:44:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500532+0000", 2024-09-16T11:44:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028087903000000001 2024-09-16T11:44:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500611+0000", 2024-09-16T11:44:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9048999999999995e-05 2024-09-16T11:44:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500676+0000", 2024-09-16T11:44:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4270000000000006e-05 2024-09-16T11:44:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19275 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 5955584~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472468+0000", 2024-09-16T11:44:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9334299819999998, 2024-09-16T11:44:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028317512, 2024-09-16T11:44:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19275 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472468+0000", 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472468+0000", 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472469+0000", 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9800000000000003e-07 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472472+0000", 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-06 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472483+0000", 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0883000000000001e-05 2024-09-16T11:44:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472497+0000", 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4158999999999999e-05 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500703+0000", 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028205480000000002 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500750+0000", 2024-09-16T11:44:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6978999999999999e-05 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500785+0000", 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5602999999999998e-05 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19276 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 6381568~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472519+0000", 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9333791639999998, 2024-09-16T11:44:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028436959000000001, 2024-09-16T11:44:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19276 2024-09-16T11:44:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472519+0000", 2024-09-16T11:44:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472519+0000", 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472519+0000", 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472526+0000", 2024-09-16T11:44:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7549999999999997e-06 2024-09-16T11:44:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472534+0000", 2024-09-16T11:44:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5700000000000004e-06 2024-09-16T11:44:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472538+0000", 2024-09-16T11:44:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0239999999999999e-06 2024-09-16T11:44:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500815+0000", 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028276881 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500890+0000", 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5693999999999997e-05 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500956+0000", 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5392e-05 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19278 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 6701056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472568+0000", 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9333298939999999, 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028494954999999999, 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19278 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472568+0000", 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472568+0000", 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472569+0000", 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472571+0000", 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9850000000000001e-06 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472577+0000", 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0549999999999996e-06 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472582+0000", 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-16T11:44:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.500982+0000", 2024-09-16T11:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028399974000000001 2024-09-16T11:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501026+0000", 2024-09-16T11:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4440000000000001e-05 2024-09-16T11:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501063+0000", 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6999999999999998e-05 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19281 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 6807552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472740+0000", 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9331578880000002, 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028491267000000001, 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19281 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472740+0000", 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472740+0000", 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472741+0000", 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0699999999999996e-07 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472745+0000", 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472754+0000", 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9220000000000003e-06 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472759+0000", 2024-09-16T11:44:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0629999999999996e-06 2024-09-16T11:44:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501091+0000", 2024-09-16T11:44:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028332053999999999 2024-09-16T11:44:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501168+0000", 2024-09-16T11:44:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7033000000000006e-05 2024-09-16T11:44:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501231+0000", 2024-09-16T11:44:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3096999999999996e-05 2024-09-16T11:44:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19282 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9330607559999997, 2024-09-16T11:44:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521151000000002, 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19282 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472838+0000", 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472848+0000", 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9799999999999993e-06 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472858+0000", 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9429999999999995e-06 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472862+0000", 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4460000000000003e-06 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501277+0000", 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028414900999999999 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501322+0000", 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4880999999999999e-05 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501358+0000", 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6245000000000003e-05 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19284 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472990+0000", 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.932907793, 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028479695999999999, 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19284 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472990+0000", 2024-09-16T11:44:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472990+0000", 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472991+0000", 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472994+0000", 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1219999999999999e-06 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473001+0000", 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4719999999999998e-06 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473006+0000", 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5040000000000004e-06 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501388+0000", 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381661999999998 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501432+0000", 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4131999999999997e-05 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501470+0000", 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8114000000000001e-05 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19285 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7872512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473074+0000", 2024-09-16T11:44:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9328235540000005, 2024-09-16T11:44:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028486999999999998, 2024-09-16T11:44:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19285 2024-09-16T11:44:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473074+0000", 2024-09-16T11:44:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473074+0000", 2024-09-16T11:44:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473075+0000", 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473078+0000", 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5679999999999998e-06 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473083+0000", 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1930000000000001e-06 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473091+0000", 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5639999999999997e-06 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501490+0000", 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028398685 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501527+0000", 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7426999999999999e-05 2024-09-16T11:44:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501561+0000", 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3994e-05 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19286 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7979008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9326878240000003, 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521860999999999, 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19286 2024-09-16T11:44:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473211+0000", 2024-09-16T11:44:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:44:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473214+0000", 2024-09-16T11:44:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1080000000000001e-06 2024-09-16T11:44:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473226+0000", 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2441e-05 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473234+0000", 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729999999999998e-06 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501592+0000", 2024-09-16T11:44:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028358491999999999 2024-09-16T11:44:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501671+0000", 2024-09-16T11:44:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8846999999999999e-05 2024-09-16T11:44:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501732+0000", 2024-09-16T11:44:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1118000000000007e-05 2024-09-16T11:44:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19288 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 8298496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473367+0000", 2024-09-16T11:44:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9325305870000005, 2024-09-16T11:44:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028460488999999999, 2024-09-16T11:44:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19288 2024-09-16T11:44:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473367+0000", 2024-09-16T11:44:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473367+0000", 2024-09-16T11:44:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473368+0000", 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473371+0000", 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1540000000000002e-06 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473386+0000", 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4430000000000001e-05 2024-09-16T11:44:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473390+0000", 2024-09-16T11:44:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999996e-06 2024-09-16T11:44:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501754+0000", 2024-09-16T11:44:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028363546999999999 2024-09-16T11:44:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501794+0000", 2024-09-16T11:44:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9965000000000001e-05 2024-09-16T11:44:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501828+0000", 2024-09-16T11:44:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3926e-05 2024-09-16T11:44:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20616 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 6545408~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560522+0000", 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8453756689999992, 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028053345, 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20616 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560522+0000", 2024-09-16T11:44:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560522+0000", 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560522+0000", 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560523+0000", 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560526+0000", 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7130000000000001e-06 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560528+0000", 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588461+0000", 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027932413999999999 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588505+0000", 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4199000000000002e-05 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588576+0000", 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0899e-05 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20617 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 6864896~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560531+0000", 2024-09-16T11:44:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8453669759999993, 2024-09-16T11:44:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028225698, 2024-09-16T11:44:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20617 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560531+0000", 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560531+0000", 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560531+0000", 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560532+0000", 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7900000000000007e-07 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560535+0000", 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7539999999999998e-06 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560537+0000", 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7349999999999999e-06 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588606+0000", 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028069436999999999 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588668+0000", 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2287999999999997e-05 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588757+0000", 2024-09-16T11:44:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8299000000000003e-05 2024-09-16T11:44:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20618 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7290880~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560539+0000", 2024-09-16T11:44:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8453590480000006, 2024-09-16T11:44:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028393137999999998, 2024-09-16T11:44:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20618 2024-09-16T11:44:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560539+0000", 2024-09-16T11:44:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560539+0000", 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560539+0000", 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560540+0000", 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9899999999999999e-07 2024-09-16T11:44:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560543+0000", 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7520000000000001e-06 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560545+0000", 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9740000000000001e-06 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588791+0000", 2024-09-16T11:44:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028245856 2024-09-16T11:44:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588864+0000", 2024-09-16T11:44:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3096000000000006e-05 2024-09-16T11:44:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588932+0000", 2024-09-16T11:44:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8281000000000006e-05 2024-09-16T11:44:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20619 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7610368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8453386559999991, 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028500152000000001, 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20619 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560560+0000", 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9499999999999998e-07 2024-09-16T11:44:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560561+0000", 2024-09-16T11:44:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.071e-06 2024-09-16T11:44:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560565+0000", 2024-09-16T11:44:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:44:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560567+0000", 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588949+0000", 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381878999999999 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588989+0000", 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0649000000000003e-05 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589059+0000", 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0239000000000003e-05 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20620 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7716864~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8453292559999994, 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028627621999999998, 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20620 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562377+0000", 2024-09-16T11:44:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018081729999999999 2024-09-16T11:44:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562380+0000", 2024-09-16T11:44:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0469999999999998e-06 2024-09-16T11:44:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562386+0000", 2024-09-16T11:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2040000000000002e-06 2024-09-16T11:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562390+0000", 2024-09-16T11:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589092+0000", 2024-09-16T11:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026701798999999998 2024-09-16T11:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589139+0000", 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7320000000000001e-05 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589196+0000", 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7201999999999999e-05 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20626 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 8142848~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.562442+0000", 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8434558669999994, 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028046026000000002, 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20626 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562442+0000", 2024-09-16T11:44:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562442+0000", 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562442+0000", 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562444+0000", 2024-09-16T11:44:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0729999999999999e-06 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562446+0000", 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8320000000000002e-06 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562448+0000", 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.590347+0000", 2024-09-16T11:44:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027898284999999998 2024-09-16T11:44:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.590417+0000", 2024-09-16T11:44:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0205999999999998e-05 2024-09-16T11:44:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.590488+0000", 2024-09-16T11:44:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1297999999999994e-05 2024-09-16T11:44:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20627 4.5 4.e2161ba5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.562450+0000", 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8434477129999998, 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028076170000000001, 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20627 2024-09-16T11:44:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562450+0000", 2024-09-16T11:44:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562450+0000", 2024-09-16T11:44:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562450+0000", 2024-09-16T11:44:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0800000000000001e-07 2024-09-16T11:44:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562452+0000", 2024-09-16T11:44:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.06e-06 2024-09-16T11:44:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562454+0000", 2024-09-16T11:44:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.932e-06 2024-09-16T11:44:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562456+0000", 2024-09-16T11:44:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:44:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.590517+0000", 2024-09-16T11:44:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028060259000000001 2024-09-16T11:44:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.590526+0000", 2024-09-16T11:44:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7929999999999992e-06 2024-09-16T11:44:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.499 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:05.499 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:44:05.499 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:44:05.499 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T11:44:05.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T11:44:05.502 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:05.503 INFO:teuthology.orchestra.run.smithi070.stdout:3719 writesame 0xec8a6ec thru 0xec914bb data_size 0x10(0x6dd0 bytes) 2024-09-16T11:44:05.503 INFO:teuthology.orchestra.run.smithi070.stdout:3720 read 0xbeb82e8 thru 0xbec69bb (0xe6d4 bytes) 2024-09-16T11:44:05.503 INFO:teuthology.orchestra.run.smithi070.stdout:3721 write 0x1113412 thru 0x111c87a (0x9469 bytes) 2024-09-16T11:44:05.503 INFO:teuthology.orchestra.run.smithi070.stdout:3726 trunc from 0xec914bc to 0x903a874 2024-09-16T11:44:05.503 INFO:teuthology.orchestra.run.smithi070.stdout:3727 read 0x92a25d thru 0x93229d (0x8041 bytes) 2024-09-16T11:44:05.504 INFO:teuthology.orchestra.run.smithi070.stdout:3729 write 0x8afbf08 thru 0x8b0515a (0x9253 bytes) 2024-09-16T11:44:05.504 INFO:teuthology.orchestra.run.smithi070.stdout:3730 trunc from 0x903a874 to 0xa198670 2024-09-16T11:44:05.504 INFO:teuthology.orchestra.run.smithi070.stdout:3731 read 0x4393599 thru 0x43a34f7 (0xff5f bytes) 2024-09-16T11:44:05.504 INFO:teuthology.orchestra.run.smithi070.stdout:3732 write 0xa57fe89 thru 0xa58f224 (0xf39c bytes) 2024-09-16T11:44:05.504 INFO:teuthology.orchestra.run.smithi070.stdout:3733 writesame 0xa6cc42b thru 0xa6da9b9 data_size 0x3(0xe58f bytes) 2024-09-16T11:44:05.504 INFO:teuthology.orchestra.run.smithi070.stdout:3734 write 0x21a7010 thru 0x21b6801 (0xf7f2 bytes) 2024-09-16T11:44:05.504 INFO:teuthology.orchestra.run.smithi070.stdout:3735 trunc from 0xa6da9ba to 0x9ce5df9 2024-09-16T11:44:05.504 INFO:teuthology.orchestra.run.smithi070.stdout:3737 punch from 0x804397c to 0x804dd99, (0xa41d bytes) 2024-09-16T11:44:05.504 INFO:teuthology.orchestra.run.smithi070.stdout:3738 read 0x74d26dc thru 0x74dcd97 (0xa6bc bytes) 2024-09-16T11:44:05.504 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:05.504 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:05.504 INFO:teuthology.orchestra.run.smithi070.stdout:3741 read 0x74c7cd2 thru 0x74d3957 (0xbc86 bytes) 2024-09-16T11:44:05.505 INFO:teuthology.orchestra.run.smithi070.stdout:3742 trunc from 0x9ce5df9 to 0x74baef8 2024-09-16T11:44:05.505 INFO:teuthology.orchestra.run.smithi070.stdout:3743 punch from 0x3b746d7 to 0x3b84533, (0xfe5c bytes) 2024-09-16T11:44:05.505 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:05.499+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:05.505 INFO:teuthology.orchestra.run.smithi070.stdout:3744 writesame 0xe10a87f thru 0xe1183b6 data_size 0xb8(0xdb38 bytes) 2024-09-16T11:44:05.506 INFO:teuthology.orchestra.run.smithi070.stdout:3746 punch from 0xab6f1d1 to 0xab73374, (0x41a3 bytes) 2024-09-16T11:44:05.509 INFO:teuthology.orchestra.run.smithi070.stdout:3748 write 0xe49ab55 thru 0xe49b4a4 (0x950 bytes) 2024-09-16T11:44:05.511 INFO:teuthology.orchestra.run.smithi070.stdout:3749 read 0x391a9f3 thru 0x3923e44 (0x9452 bytes) 2024-09-16T11:44:05.513 INFO:teuthology.orchestra.run.smithi070.stdout:3750 trunc from 0xe49b4a5 to 0xc9c75f7 2024-09-16T11:44:05.523 INFO:teuthology.orchestra.run.smithi070.stdout:3752 write 0x32db25 thru 0x33a07f (0xc55b bytes) 2024-09-16T11:44:05.525 INFO:teuthology.orchestra.run.smithi070.stdout:3753 read 0x970315c thru 0x970c9a8 (0x984d bytes) 2024-09-16T11:44:05.525 INFO:teuthology.orchestra.run.smithi070.stdout:3754 write 0x7358c7 thru 0x73c3ea (0x6b24 bytes) 2024-09-16T11:44:05.526 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:05.526 INFO:teuthology.orchestra.run.smithi070.stdout:3756 write 0x914969 thru 0x918579 (0x3c11 bytes) 2024-09-16T11:44:05.533 INFO:teuthology.orchestra.run.smithi070.stdout:3758 write 0xd45c290 thru 0xd45eb63 (0x28d4 bytes) 2024-09-16T11:44:05.536 INFO:teuthology.orchestra.run.smithi070.stdout:3759 write 0x51467c5 thru 0x514bcab (0x54e7 bytes) 2024-09-16T11:44:05.538 INFO:teuthology.orchestra.run.smithi070.stdout:3760 read 0x26a9e74 thru 0x26b89ec (0xeb79 bytes) 2024-09-16T11:44:05.539 INFO:teuthology.orchestra.run.smithi070.stdout:3761 writesame 0x466d0e8 thru 0x4672fdf data_size 0xc(0x5ef8 bytes) 2024-09-16T11:44:05.542 INFO:teuthology.orchestra.run.smithi070.stdout:3765 write 0xc504ab3 thru 0xc509e87 (0x53d5 bytes) 2024-09-16T11:44:05.545 INFO:teuthology.orchestra.run.smithi070.stdout:3766 clone 23 order 19 su 8192 sc 12 2024-09-16T11:44:05.602 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:05.600+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:05.605 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:05.605 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:05.613 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:05.703 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:05.701+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:05.752 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:05.752 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:05.752 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:05.752 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:05.752 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:05.760 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:05.773 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:05.773 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:05.773 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 56, 2024-09-16T11:44:05.773 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "3420bcad-625f-4e15-aaff-fb4236322168", 2024-09-16T11:44:05.773 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2024-09-16T11:43:01.654626+0000", 2024-09-16T11:44:05.773 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2024-09-16T11:44:04.886301+0000", 2024-09-16T11:44:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2024-09-16T11:43:11.689589+0000", 2024-09-16T11:44:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2024-09-16T11:43:04.526236+0000", 2024-09-16T11:44:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T11:44:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2024-09-16T11:44:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2024-09-16T11:44:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2024-09-16T11:44:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2024-09-16T11:44:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2024-09-16T11:44:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2024-09-16T11:44:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 5, 2024-09-16T11:44:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T11:44:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T11:44:05.775 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T11:44:05.775 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2024-09-16T11:44:05.775 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 4, 2024-09-16T11:44:05.775 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2024-09-16T11:44:05.775 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2024-09-16T11:44:05.775 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2024-09-16T11:44:05.775 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "quincy", 2024-09-16T11:44:05.775 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2024-09-16T11:44:05.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.775 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2024-09-16T11:44:05.775 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2024-09-16T11:44:05.775 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T11:43:11.406582+0000", 2024-09-16T11:44:05.775 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T11:44:05.776 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T11:44:05.776 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T11:44:05.776 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-16T11:44:05.776 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-16T11:44:05.776 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T11:44:05.776 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:44:05.776 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:44:05.776 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:44:05.776 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:44:05.776 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T11:44:05.776 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:44:05.776 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2024-09-16T11:44:05.776 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2024-09-16T11:44:05.777 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2024-09-16T11:44:05.777 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2024-09-16T11:44:05.777 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2024-09-16T11:44:05.777 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T11:44:05.777 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T11:44:05.777 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T11:44:05.777 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T11:44:05.777 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T11:44:05.777 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T11:44:05.777 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T11:44:05.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.777 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "16", 2024-09-16T11:44:05.777 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T11:44:05.778 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:44:05.778 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:44:05.778 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T11:44:05.778 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:44:05.778 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T11:44:05.778 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T11:44:05.778 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T11:44:05.778 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T11:44:05.778 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T11:44:05.778 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T11:44:05.778 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T11:44:05.778 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T11:44:05.778 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.779 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2024-09-16T11:44:05.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.781 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T11:44:05.781 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2024-09-16T11:44:05.781 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.781 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2024-09-16T11:44:05.781 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2024-09-16T11:44:05.781 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T11:43:13.125709+0000", 2024-09-16T11:44:05.781 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T11:44:05.781 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T11:44:05.781 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T11:44:05.781 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-16T11:44:05.781 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T11:44:05.782 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T11:44:05.783 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T11:44:05.783 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T11:44:05.783 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T11:44:05.783 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T11:44:05.783 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T11:44:05.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.783 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "23", 2024-09-16T11:44:05.783 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "23", 2024-09-16T11:44:05.783 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "23", 2024-09-16T11:44:05.783 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "23", 2024-09-16T11:44:05.784 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T11:44:05.784 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:44:05.784 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 3, 2024-09-16T11:44:05.784 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 21, 2024-09-16T11:44:05.784 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T11:44:05.784 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T11:44:05.784 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T11:44:05.784 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T11:44:05.784 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [ 2024-09-16T11:44:05.784 INFO:teuthology.orchestra.run.smithi070.stdout: 3 2024-09-16T11:44:05.784 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:05.784 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": 3, 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": 3, 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T11:44:05.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.786 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T11:44:05.786 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T11:44:05.786 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T11:44:05.786 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:44:05.786 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:44:05.786 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:44:05.786 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:44:05.786 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T11:44:05.786 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T11:44:05.786 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T11:44:05.786 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T11:44:05.786 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T11:44:05.787 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T11:44:05.787 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T11:44:05.787 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.787 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 3, 2024-09-16T11:44:05.787 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cache", 2024-09-16T11:44:05.787 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T11:43:21.227888+0000", 2024-09-16T11:44:05.787 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8201, 2024-09-16T11:44:05.787 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,incomplete_clones,selfmanaged_snaps", 2024-09-16T11:44:05.787 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T11:44:05.787 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-16T11:44:05.788 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-16T11:44:05.788 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T11:44:05.788 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:44:05.788 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:44:05.788 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:44:05.788 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:44:05.788 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T11:44:05.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:44:05.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 4, 2024-09-16T11:44:05.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 4, 2024-09-16T11:44:05.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 4, 2024-09-16T11:44:05.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 4, 2024-09-16T11:44:05.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 4, 2024-09-16T11:44:05.789 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T11:44:05.789 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T11:44:05.789 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T11:44:05.789 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T11:44:05.789 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T11:44:05.789 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T11:44:05.789 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T11:44:05.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.789 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "27", 2024-09-16T11:44:05.789 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "23", 2024-09-16T11:44:05.789 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "23", 2024-09-16T11:44:05.792 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "23", 2024-09-16T11:44:05.792 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T11:44:05.792 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:44:05.792 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 3, 2024-09-16T11:44:05.792 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 21, 2024-09-16T11:44:05.792 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T11:44:05.792 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T11:44:05.792 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T11:44:05.792 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T11:44:05.792 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T11:44:05.793 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": 2, 2024-09-16T11:44:05.793 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T11:44:05.793 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T11:44:05.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "writeback", 2024-09-16T11:44:05.793 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T11:44:05.793 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 250, 2024-09-16T11:44:05.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:44:05.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:44:05.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:44:05.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T11:44:05.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T11:44:05.793 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bloom", 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: "false_positive_probability": 0.050000000000000003, 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size": 0, 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: "seed": 0 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 60, 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 8, 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:44:05.794 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [ 2024-09-16T11:44:05.795 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:05.795 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:05.795 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:05.795 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:05.795 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:05.795 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:05.795 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:05.795 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000 2024-09-16T11:44:05.795 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:05.795 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T11:44:05.795 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T11:44:05.795 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T11:44:05.795 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T11:44:05.796 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T11:44:05.796 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T11:44:05.796 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.796 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 4, 2024-09-16T11:44:05.796 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "pool_client.0", 2024-09-16T11:44:05.796 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T11:43:30.105533+0000", 2024-09-16T11:44:05.796 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T11:44:05.796 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T11:44:05.796 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T11:44:05.796 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-16T11:44:05.796 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-16T11:44:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T11:44:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:44:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:44:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:44:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:44:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T11:44:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:44:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T11:44:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T11:44:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T11:44:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T11:44:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T11:44:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "56", 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:44:05.798 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 25, 2024-09-16T11:44:05.799 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 56, 2024-09-16T11:44:05.799 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T11:44:05.799 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T11:44:05.799 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T11:44:05.799 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T11:44:05.799 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T11:44:05.799 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T11:44:05.799 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T11:44:05.799 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T11:44:05.799 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T11:44:05.799 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T11:44:05.799 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T11:44:05.799 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:44:05.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:44:05.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:44:05.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T11:44:05.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T11:44:05.800 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T11:44:05.800 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T11:44:05.800 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T11:44:05.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.800 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T11:44:05.800 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T11:44:05.800 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T11:44:05.800 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:44:05.800 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:44:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:44:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:44:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T11:44:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T11:44:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T11:44:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T11:44:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T11:44:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T11:44:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T11:44:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "91f052e4-4c8f-4484-92ce-bd1a7f8f3efb", 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:05.803 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.804 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.804 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2024-09-16T11:44:05.804 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:05.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.804 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.804 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2024-09-16T11:44:05.804 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:05.804 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.804 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.804 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:05.805 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.805 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.805 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2024-09-16T11:44:05.805 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:05.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.805 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.805 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2024-09-16T11:44:05.805 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:05.805 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.805 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.806 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:05.806 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.806 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.806 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2024-09-16T11:44:05.806 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:05.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.806 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.806 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2024-09-16T11:44:05.806 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:05.806 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.806 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/33355", 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/33355", 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/33355", 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/33355", 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "568b1280-7365-4094-afe8-b0b5e766de6f", 2024-09-16T11:44:05.807 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:05.808 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:05.808 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:05.808 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:05.808 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:05.808 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:05.808 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 14, 2024-09-16T11:44:05.808 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:05.808 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:05.808 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:05.808 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:05.808 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.808 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.809 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.810 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2024-09-16T11:44:05.810 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:05.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.810 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.810 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2024-09-16T11:44:05.810 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:05.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.810 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.810 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:05.810 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.812 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.812 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2024-09-16T11:44:05.812 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:05.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.812 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.812 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2024-09-16T11:44:05.812 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:05.812 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.812 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.812 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/33359", 2024-09-16T11:44:05.812 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/33359", 2024-09-16T11:44:05.813 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/33359", 2024-09-16T11:44:05.813 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/33359", 2024-09-16T11:44:05.813 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:05.813 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:05.813 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:05.813 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.813 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T11:44:05.813 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "328adfa3-c1b4-4745-9759-7327c8cbd499", 2024-09-16T11:44:05.813 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:05.813 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:05.813 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:05.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.815 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.815 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2024-09-16T11:44:05.815 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:05.815 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.815 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.815 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:05.815 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.815 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.815 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2024-09-16T11:44:05.815 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:05.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.816 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.816 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2024-09-16T11:44:05.816 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:05.816 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.816 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.816 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:05.816 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.816 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.816 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2024-09-16T11:44:05.817 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:05.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.817 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.817 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2024-09-16T11:44:05.817 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:05.817 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.817 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.817 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:05.817 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.817 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/33356", 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/33356", 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6815/33356", 2024-09-16T11:44:05.818 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/33356", 2024-09-16T11:44:05.819 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:05.819 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:05.819 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:05.819 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.819 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T11:44:05.819 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "6df66ae7-e835-4d8f-bd27-243d7a9705b5", 2024-09-16T11:44:05.819 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:05.819 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:05.819 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:05.819 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:05.819 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:05.820 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:05.820 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 14, 2024-09-16T11:44:05.820 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:05.820 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:05.820 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:05.820 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:05.820 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.820 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.820 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2024-09-16T11:44:05.820 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:05.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.821 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.821 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2024-09-16T11:44:05.821 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:05.821 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.821 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.821 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:05.821 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.821 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.821 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2024-09-16T11:44:05.821 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:05.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:05.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.823 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.823 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2024-09-16T11:44:05.823 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:05.823 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.823 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.823 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:05.823 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.823 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.823 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2024-09-16T11:44:05.823 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:05.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.824 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.824 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2024-09-16T11:44:05.824 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:05.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.824 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.824 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/33384", 2024-09-16T11:44:05.824 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/33384", 2024-09-16T11:44:05.824 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/33384", 2024-09-16T11:44:05.824 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/33384", 2024-09-16T11:44:05.824 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "29f52d62-d4c3-49f9-992f-23817f5622d2", 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:05.825 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T11:44:05.826 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:05.826 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:05.826 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:05.826 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:05.826 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.826 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.826 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6800", 2024-09-16T11:44:05.826 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:05.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.826 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.826 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6803", 2024-09-16T11:44:05.827 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:05.827 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.827 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.827 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:05.827 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.827 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.827 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6805", 2024-09-16T11:44:05.827 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:05.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.827 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.828 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6807", 2024-09-16T11:44:05.828 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:05.828 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.828 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.828 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:05.828 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.828 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.828 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6813", 2024-09-16T11:44:05.828 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:05.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6815", 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6809", 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.830 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.830 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6811", 2024-09-16T11:44:05.830 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:05.830 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.830 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.192:6803/33331", 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.192:6807/33331", 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.192:6815/33331", 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.192:6811/33331", 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "0c1a275a-3c75-4e0b-bd35-d34b64d99c0a", 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:05.831 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 14, 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6816", 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:05.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6817", 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6818", 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:05.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6819", 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6822", 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:05.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6823", 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6820", 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:05.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.836 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.836 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6821", 2024-09-16T11:44:05.836 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:05.836 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.836 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.836 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.192:6817/33333", 2024-09-16T11:44:05.836 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.192:6819/33333", 2024-09-16T11:44:05.836 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.192:6823/33333", 2024-09-16T11:44:05.836 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.192:6821/33333", 2024-09-16T11:44:05.836 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:05.836 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T11:44:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "d79f0070-ce58-48e7-a56a-2762ec76a271", 2024-09-16T11:44:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6801", 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6802", 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:05.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6804", 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6806", 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:05.839 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6812", 2024-09-16T11:44:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6814", 2024-09-16T11:44:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6808", 2024-09-16T11:44:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6810", 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.192:6802/33332", 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.192:6806/33332", 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.192:6814/33332", 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.192:6810/33332", 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.843 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T11:44:05.843 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "537fa5df-3d0a-4dd4-8155-20ee94e41c82", 2024-09-16T11:44:05.843 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:05.843 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:05.843 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:05.843 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:05.843 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:05.843 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:05.843 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 14, 2024-09-16T11:44:05.843 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 16, 2024-09-16T11:44:05.843 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:05.843 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:05.843 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6824", 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6825", 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.844 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:05.845 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.845 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.845 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6826", 2024-09-16T11:44:05.845 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:05.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.845 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.845 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6827", 2024-09-16T11:44:05.845 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:05.845 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.845 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.846 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:05.846 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.846 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.846 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6830", 2024-09-16T11:44:05.846 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:05.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.846 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.846 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6831", 2024-09-16T11:44:05.846 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:05.846 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.846 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.847 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:05.847 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:05.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.847 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:05.847 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6828", 2024-09-16T11:44:05.847 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:05.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.847 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:05.847 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6829", 2024-09-16T11:44:05.847 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:05.847 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.848 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.848 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.192:6825/33362", 2024-09-16T11:44:05.848 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.192:6827/33362", 2024-09-16T11:44:05.848 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.192:6831/33362", 2024-09-16T11:44:05.848 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.192:6829/33362", 2024-09-16T11:44:05.848 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:05.848 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:05.848 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:05.848 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:05.848 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.848 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:05.848 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.581514+0000", 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:05.849 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.674271+0000", 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:05.850 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:10.545402+0000", 2024-09-16T11:44:05.851 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:05.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.851 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T11:44:05.851 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:05.851 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:05.851 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:05.851 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:05.851 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:05.851 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.733565+0000", 2024-09-16T11:44:05.851 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:05.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.852 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T11:44:05.852 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:05.852 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:05.852 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:05.852 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:05.852 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:05.852 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.623479+0000", 2024-09-16T11:44:05.852 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:05.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.852 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T11:44:05.852 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:05.852 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.730066+0000", 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:05.853 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:10.556650+0000", 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.702368+0000", 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:05.854 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2024-09-16T11:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2024-09-16T11:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2024-09-16T11:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2024-09-16T11:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2024-09-16T11:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2024-09-16T11:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2024-09-16T11:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2024-09-16T11:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2024-09-16T11:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2024-09-16T11:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2024-09-16T11:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:05.856 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:05.858 INFO:tasks.thrashosds.thrasher:j is {'epoch': 56, 'fsid': '3420bcad-625f-4e15-aaff-fb4236322168', 'created': '2024-09-16T11:43:01.654626+0000', 'modified': '2024-09-16T11:44:04.886301+0000', 'last_up_change': '2024-09-16T11:43:11.689589+0000', 'last_in_change': '2024-09-16T11:43:04.526236+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T11:43:11.406582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T11:43:13.125709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '23', 'last_force_op_resend_prenautilus': '23', 'last_force_op_resend_preluminous': '23', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-09-16T11:43:21.227888+0000', 'flags': 8201, 'flags_names': 'hashpspool,incomplete_clones,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '23', 'last_force_op_resend_prenautilus': '23', 'last_force_op_resend_preluminous': '23', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T11:43:30.105533+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 56, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': '91f052e4-4c8f-4484-92ce-bd1a7f8f3efb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6800', 'nonce': 33355}, {'type': 'v1', 'addr': '172.21.15.70:6801', 'nonce': 33355}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6802', 'nonce': 33355}, {'type': 'v1', 'addr': '172.21.15.70:6803', 'nonce': 33355}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6806', 'nonce': 33355}, {'type': 'v1', 'addr': '172.21.15.70:6807', 'nonce': 33355}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6804', 'nonce': 33355}, {'type': 'v1', 'addr': '172.21.15.70:6805', 'nonce': 33355}]}, 'public_addr': '172.21.15.70:6801/33355', 'cluster_addr': '172.21.15.70:6803/33355', 'heartbeat_back_addr': '172.21.15.70:6807/33355', 'heartbeat_front_addr': '172.21.15.70:6805/33355', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '568b1280-7365-4094-afe8-b0b5e766de6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6816', 'nonce': 33359}, {'type': 'v1', 'addr': '172.21.15.70:6817', 'nonce': 33359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6818', 'nonce': 33359}, {'type': 'v1', 'addr': '172.21.15.70:6819', 'nonce': 33359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6822', 'nonce': 33359}, {'type': 'v1', 'addr': '172.21.15.70:6823', 'nonce': 33359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6820', 'nonce': 33359}, {'type': 'v1', 'addr': '172.21.15.70:6821', 'nonce': 33359}]}, 'public_addr': '172.21.15.70:6817/33359', 'cluster_addr': '172.21.15.70:6819/33359', 'heartbeat_back_addr': '172.21.15.70:6823/33359', 'heartbeat_front_addr': '172.21.15.70:6821/33359', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '328adfa3-c1b4-4745-9759-7327c8cbd499', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6808', 'nonce': 33356}, {'type': 'v1', 'addr': '172.21.15.70:6809', 'nonce': 33356}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6810', 'nonce': 33356}, {'type': 'v1', 'addr': '172.21.15.70:6811', 'nonce': 33356}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6814', 'nonce': 33356}, {'type': 'v1', 'addr': '172.21.15.70:6815', 'nonce': 33356}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6812', 'nonce': 33356}, {'type': 'v1', 'addr': '172.21.15.70:6813', 'nonce': 33356}]}, 'public_addr': '172.21.15.70:6809/33356', 'cluster_addr': '172.21.15.70:6811/33356', 'heartbeat_back_addr': '172.21.15.70:6815/33356', 'heartbeat_front_addr': '172.21.15.70:6813/33356', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '6df66ae7-e835-4d8f-bd27-243d7a9705b5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6824', 'nonce': 33384}, {'type': 'v1', 'addr': '172.21.15.70:6825', 'nonce': 33384}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6826', 'nonce': 33384}, {'type': 'v1', 'addr': '172.21.15.70:6827', 'nonce': 33384}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6830', 'nonce': 33384}, {'type': 'v1', 'addr': '172.21.15.70:6831', 'nonce': 33384}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6828', 'nonce': 33384}, {'type': 'v1', 'addr': '172.21.15.70:6829', 'nonce': 33384}]}, 'public_addr': '172.21.15.70:6825/33384', 'cluster_addr': '172.21.15.70:6827/33384', 'heartbeat_back_addr': '172.21.15.70:6831/33384', 'heartbeat_front_addr': '172.21.15.70:6829/33384', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '29f52d62-d4c3-49f9-992f-23817f5622d2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 33331}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 33331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6805', 'nonce': 33331}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 33331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6813', 'nonce': 33331}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 33331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6809', 'nonce': 33331}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 33331}]}, 'public_addr': '172.21.15.192:6803/33331', 'cluster_addr': '172.21.15.192:6807/33331', 'heartbeat_back_addr': '172.21.15.192:6815/33331', 'heartbeat_front_addr': '172.21.15.192:6811/33331', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0c1a275a-3c75-4e0b-bd35-d34b64d99c0a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 33333}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 33333}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 33333}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 33333}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 33333}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 33333}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 33333}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 33333}]}, 'public_addr': '172.21.15.192:6817/33333', 'cluster_addr': '172.21.15.192:6819/33333', 'heartbeat_back_addr': '172.21.15.192:6823/33333', 'heartbeat_front_addr': '172.21.15.192:6821/33333', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd79f0070-ce58-48e7-a56a-2762ec76a271', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6801', 'nonce': 33332}, {'type': 'v1', 'addr': '172.21.15.192:6802', 'nonce': 33332}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 33332}, {'type': 'v1', 'addr': '172.21.15.192:6806', 'nonce': 33332}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 33332}, {'type': 'v1', 'addr': '172.21.15.192:6814', 'nonce': 33332}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 33332}, {'type': 'v1', 'addr': '172.21.15.192:6810', 'nonce': 33332}]}, 'public_addr': '172.21.15.192:6802/33332', 'cluster_addr': '172.21.15.192:6806/33332', 'heartbeat_back_addr': '172.21.15.192:6814/33332', 'heartbeat_front_addr': '172.21.15.192:6810/33332', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '537fa5df-3d0a-4dd4-8155-20ee94e41c82', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 16, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6824', 'nonce': 33362}, {'type': 'v1', 'addr': '172.21.15.192:6825', 'nonce': 33362}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6826', 'nonce': 33362}, {'type': 'v1', 'addr': '172.21.15.192:6827', 'nonce': 33362}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6830', 'nonce': 33362}, {'type': 'v1', 'addr': '172.21.15.192:6831', 'nonce': 33362}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6828', 'nonce': 33362}, {'type': 'v1', 'addr': '172.21.15.192:6829', 'nonce': 33362}]}, 'public_addr': '172.21.15.192:6825/33362', 'cluster_addr': '172.21.15.192:6827/33362', 'heartbeat_back_addr': '172.21.15.192:6831/33362', 'heartbeat_front_addr': '172.21.15.192:6829/33362', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.581514+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.674271+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:10.545402+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.733565+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.623479+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.730066+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:10.556650+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.702368+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T11:44:05.858 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:44:05.859 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:05.855+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:05.860 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:05.860 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:05.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:05.860 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:05.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.860 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:05.861 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:05.861 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.937341979999999, 2024-09-16T11:44:05.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:44:05.861 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.861 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:05.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:05.861 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:44:05.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.861 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:44:05.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:44:05.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:05.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:44:05.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:44:05.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:05.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:44:05.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:44:05.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:05.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:44:05.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:44:05.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:44:05.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:44:05.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:44:05.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:44:05.866 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.866 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.866 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.937296287999999, 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:05.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:05.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:44:05.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:44:05.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:44:05.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:44:05.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:44:05.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:44:05.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:44:05.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:44:05.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:44:05.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:44:05.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.872 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.873 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:05.873 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:05.873 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.937239357999999, 2024-09-16T11:44:05.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:44:05.873 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.873 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:05.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:44:05.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:44:05.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:44:05.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:44:05.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:44:05.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:05.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:44:05.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:44:05.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:44:05.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:44:05.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:44:05.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:44:05.878 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.937213990000004, 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:05.879 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:44:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:05.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:44:05.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:44:05.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:44:05.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:44:05.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:44:05.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:44:05.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:44:05.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:44:05.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:44:05.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:05.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:44:05.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:44:05.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:44:05.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:44:05.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:44:05.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.937162022000003, 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:05.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:44:05.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:44:05.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:44:05.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:44:05.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:44:05.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:44:05.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:05.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:44:05.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:44:05.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:05.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:44:05.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:44:05.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:44:05.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.937080911999999, 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:05.891 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:44:05.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.892 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:05.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:05.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:44:05.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:44:05.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:44:05.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:44:05.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:44:05.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.896 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.897 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.897 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:05.897 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:05.897 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.937054934000003, 2024-09-16T11:44:05.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:44:05.897 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.897 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:05.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:44:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:44:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:44:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:44:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:05.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:44:05.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:05.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:44:05.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:44:05.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:44:05.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:44:05.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:05.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:44:05.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:44:05.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:05.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:44:05.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:44:05.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:44:05.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:44:05.903 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.903 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.903 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.903 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:05.903 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:05.903 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.519536309000003, 2024-09-16T11:44:05.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:44:05.903 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.903 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:05.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:44:05.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:44:05.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:44:05.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:44:05.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:44:05.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:44:05.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:44:05.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:44:05.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:44:05.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.909 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.909 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:05.909 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:05.909 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.519514377999997, 2024-09-16T11:44:05.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:44:05.909 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.909 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:44:05.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:44:05.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:44:05.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:05.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:44:05.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:44:05.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:05.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:44:05.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:44:05.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:44:05.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.519437490000001, 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:05.915 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:44:05.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.916 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:05.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:05.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:44:05.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:44:05.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:44:05.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:44:05.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:44:05.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:44:05.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:44:05.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:44:05.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:44:05.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:44:05.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:05.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:44:05.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:44:05.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:44:05.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:44:05.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:44:05.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28187 4.17 e52/28 4:ee916bd9:::rbd_data.11c1ce740b46.0000000000000000:head v 52'195, mlcod=52'193)", 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.3480181040000003, 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075854429999999999, 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509349+0000", 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2300000000000001e-06 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513315+0000", 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039660590000000001 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513328+0000", 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.308e-05 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513334+0000", 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:05.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513385+0000", 2024-09-16T11:44:05.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1347000000000002e-05 2024-09-16T11:44:05.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513407+0000", 2024-09-16T11:44:05.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1852999999999999e-05 2024-09-16T11:44:05.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516882+0000", 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034749690000000001 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516933+0000", 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0952999999999998e-05 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28959 4.8 e53/28 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head v 53'408, mlcod=53'406)", 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:05.925 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.8258690480000004, 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088740420000000004, 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031498+0000", 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.652e-06 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033515+0000", 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020165930000000001 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033539+0000", 2024-09-16T11:44:05.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4295999999999999e-05 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033548+0000", 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1239999999999994e-06 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033610+0000", 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1371999999999994e-05 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033637+0000", 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7098999999999999e-05 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040341+0000", 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067045819999999997 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040371+0000", 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9323999999999999e-05 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.930 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32303 4.8 e53/28 4:1436ac82:::rbd_data.11c1cab081b3.0000000000000022:head v 53'417, mlcod=53'415)", 2024-09-16T11:44:05.930 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:05.930 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.3554430740000001, 2024-09-16T11:44:05.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085565160000000001, 2024-09-16T11:44:05.930 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.931 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.931 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:05.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:05.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-09-16T11:44:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501930+0000", 2024-09-16T11:44:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7649999999999996e-06 2024-09-16T11:44:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501946+0000", 2024-09-16T11:44:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5301000000000001e-05 2024-09-16T11:44:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501952+0000", 2024-09-16T11:44:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:44:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502100+0000", 2024-09-16T11:44:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014847600000000001 2024-09-16T11:44:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502120+0000", 2024-09-16T11:44:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9303000000000001e-05 2024-09-16T11:44:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510436+0000", 2024-09-16T11:44:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083166420000000008 2024-09-16T11:44:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510479+0000", 2024-09-16T11:44:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2979000000000003e-05 2024-09-16T11:44:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32306 4.1e e53/28 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head v 53'124, mlcod=53'122)", 2024-09-16T11:44:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.3553770800000002, 2024-09-16T11:44:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084990900000000008, 2024-09-16T11:44:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:44:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501994+0000", 2024-09-16T11:44:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:44:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502002+0000", 2024-09-16T11:44:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3839999999999997e-06 2024-09-16T11:44:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502007+0000", 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6290000000000001e-06 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502454+0000", 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00044736800000000002 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502471+0000", 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6648999999999999e-05 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510455+0000", 2024-09-16T11:44:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0079841909999999999 2024-09-16T11:44:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510488+0000", 2024-09-16T11:44:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2948999999999999e-05 2024-09-16T11:44:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32307 4.8 e53/28 4:11c1c37d:::rbd_data.11c1cab081b3.0000000000000026:head v 53'418, mlcod=53'415)", 2024-09-16T11:44:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.3546217450000002, 2024-09-16T11:44:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00776212, 2024-09-16T11:44:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502750+0000", 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502760+0000", 2024-09-16T11:44:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:44:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502767+0000", 2024-09-16T11:44:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4219999999999997e-06 2024-09-16T11:44:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502802+0000", 2024-09-16T11:44:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5089999999999998e-05 2024-09-16T11:44:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502816+0000", 2024-09-16T11:44:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4092e-05 2024-09-16T11:44:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510483+0000", 2024-09-16T11:44:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00766704 2024-09-16T11:44:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510506+0000", 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3195999999999999e-05 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33583 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'416, mlcod=53'411)", 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.646663336, 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078132459999999994, 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210703+0000", 2024-09-16T11:44:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210706+0000", 2024-09-16T11:44:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-06 2024-09-16T11:44:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210712+0000", 2024-09-16T11:44:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:44:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210715+0000", 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210736+0000", 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0293999999999999e-05 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210746+0000", 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0178e-05 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218477+0000", 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007731501 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218516+0000", 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8148000000000001e-05 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33584 4.0 e53/28 4:044e4d16:::rbd_data.11c1cab081b3.0000000000000019:head v 53'176, mlcod=53'174)", 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.6458518529999999, 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070059470000000002, 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.37e-07 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211524+0000", 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4830000000000005e-06 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211535+0000", 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0842000000000001e-05 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211539+0000", 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9740000000000004e-06 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211584+0000", 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5655000000000002e-05 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211602+0000", 2024-09-16T11:44:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7756000000000001e-05 2024-09-16T11:44:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218497+0000", 2024-09-16T11:44:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0068944590000000004 2024-09-16T11:44:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218520+0000", 2024-09-16T11:44:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3141000000000001e-05 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33590 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'419, mlcod=53'415)", 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.6356051950000001, 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082499879999999998, 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221761+0000", 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221765+0000", 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1590000000000003e-06 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221774+0000", 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7490000000000004e-06 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221778+0000", 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0419999999999996e-06 2024-09-16T11:44:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221800+0000", 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1926999999999999e-05 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221810+0000", 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0079e-05 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.229980+0000", 2024-09-16T11:44:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081698469999999992 2024-09-16T11:44:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230010+0000", 2024-09-16T11:44:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0746e-05 2024-09-16T11:44:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33591 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'420, mlcod=53'415)", 2024-09-16T11:44:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.6350437109999998, 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007702576, 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222326+0000", 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222337+0000", 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1008e-05 2024-09-16T11:44:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222342+0000", 2024-09-16T11:44:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0050000000000004e-06 2024-09-16T11:44:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222366+0000", 2024-09-16T11:44:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3696000000000001e-05 2024-09-16T11:44:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222385+0000", 2024-09-16T11:44:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-16T11:44:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230012+0000", 2024-09-16T11:44:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076270940000000001 2024-09-16T11:44:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230025+0000", 2024-09-16T11:44:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2112e-05 2024-09-16T11:44:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33593 4.1e e53/28 4:7e77cec2:::rbd_data.11c1cab081b3.000000000000000c:head v 53'133, mlcod=53'131)", 2024-09-16T11:44:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.6304097889999998, 2024-09-16T11:44:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070516270000000004, 2024-09-16T11:44:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226957+0000", 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227014+0000", 2024-09-16T11:44:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7074999999999998e-05 2024-09-16T11:44:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227024+0000", 2024-09-16T11:44:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0617e-05 2024-09-16T11:44:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227030+0000", 2024-09-16T11:44:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3279999999999996e-06 2024-09-16T11:44:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227061+0000", 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1718999999999999e-05 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227074+0000", 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2738000000000001e-05 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.233975+0000", 2024-09-16T11:44:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0069008480000000002 2024-09-16T11:44:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.234008+0000", 2024-09-16T11:44:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2614000000000003e-05 2024-09-16T11:44:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:05.966 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:05.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:44:05.968 INFO:teuthology.orchestra.run.smithi070.stdout:flattening image image_client.0-clone22 2024-09-16T11:44:05.969 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:05.965+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:05.969 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:06.068 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:06.066+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:06.120 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:06.121 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:06.121 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:06.121 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:06.129 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:06.137 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-16T11:44:06.169 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:06.167+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:06.247 INFO:teuthology.orchestra.run.smithi070.stdout:3767 write 0xa02d421 thru 0xa038995 (0xb575 bytes) 2024-09-16T11:44:06.266 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:06.266 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:44:06.267 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:06.267 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:06.267 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:06.267 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:06.267 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:06.269 INFO:teuthology.orchestra.run.smithi070.stdout:3768 write 0xded730f thru 0xdedb49e (0x4190 bytes) 2024-09-16T11:44:06.270 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:06.267+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:06.274 INFO:teuthology.orchestra.run.smithi070.stdout:3769 punch from 0x7919429 to 0x7925c4f, (0xc826 bytes) 2024-09-16T11:44:06.274 INFO:teuthology.orchestra.run.smithi070.stdout:3770 read 0x51f5441 thru 0x51f7f31 (0x2af1 bytes) 2024-09-16T11:44:06.274 INFO:teuthology.orchestra.run.smithi070.stdout:3772 read 0x19ba115 thru 0x19c8cfa (0xebe6 bytes) 2024-09-16T11:44:06.276 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:06.280 INFO:teuthology.orchestra.run.smithi070.stdout:3775 trunc from 0xdedb49f to 0x94ba19a 2024-09-16T11:44:06.283 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-16T11:44:05.894496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634816258,"num_objects":1186,"num_object_clones":120,"num_object_copies":2372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1185,"num_whiteouts":797,"num_read":57223,"num_read_kb":198136,"num_write":9527,"num_write_kb":141857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8562,"ondisk_log_size":8562,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2455780,"kb_used_data":18660,"kb_used_omap":13,"kb_used_meta":2436978,"kb_avail":752518940,"statfs":{"total":773094113280,"available":770579394560,"internally_reserved":0,"allocated":19107840,"data_stored":113178397,"data_compressed":4039986,"data_compressed_allocated":10600448,"data_compressed_original":109012298,"omap_allocated":13632,"internal_metadata":2495466176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":117255773,"num_objects":410,"num_object_clones":33,"num_object_copies":820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":409,"num_whiteouts":334,"num_read":16380,"num_read_kb":76382,"num_write":2481,"num_write_kb":58723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.514690"},"pg_stats":[{"pgid":"4.1f","version":"55'175","reported_seq":1709,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:04.929220+0000","last_change":"2024-09-16T11:43:33.782279+0000","last_active":"2024-09-16T11:44:04.929220+0000","last_peered":"2024-09-16T11:44:04.929220+0000","last_clean":"2024-09-16T11:44:04.929220+0000","last_became_active":"2024-09-16T11:43:31.779428+0000","last_became_peered":"2024-09-16T11:43:31.779428+0000","last_unstale":"2024-09-16T11:44:04.929220+0000","last_undegraded":"2024-09-16T11:44:04.929220+0000","last_fullsized":"2024-09-16T11:44:04.929220+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:30.437698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090324899999999996,"stat_sum":{"num_bytes":11497079,"num_objects":26,"num_object_clones":2,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":15,"num_read":2127,"num_read_kb":3619,"num_write":171,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"54'158","reported_seq":910,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T11:44:04.054827+0000","last_change":"2024-09-16T11:43:34.248024+0000","last_active":"2024-09-16T11:44:04.054827+0000","last_peered":"2024-09-16T11:44:04.054827+0000","last_clean":"2024-09-16T11:44:04.054827+0000","last_became_active":"2024-09-16T11:43:31.889706+0000","last_became_peered":"2024-09-16T11:43:31.889706+0000","last_unstale":"2024-09-16T11:44:04.054827+0000","last_undegraded":"2024-09-16T11:44:04.054827+0000","last_fullsized":"2024-09-16T11:44:04.054827+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:25.242116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027516299999999998,"stat_sum":{"num_bytes":15612391,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":23,"num_read":839,"num_read_kb":5426,"num_write":148,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"52'123","reported_seq":940,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:01.196592+0000","last_change":"2024-09-16T11:43:33.777888+0000","last_active":"2024-09-16T11:44:01.196592+0000","last_peered":"2024-09-16T11:44:01.196592+0000","last_clean":"2024-09-16T11:44:01.196592+0000","last_became_active":"2024-09-16T11:43:31.775021+0000","last_became_peered":"2024-09-16T11:43:31.775021+0000","last_unstale":"2024-09-16T11:44:01.196592+0000","last_undegraded":"2024-09-16T11:44:01.196592+0000","last_fullsized":"2024-09-16T11:44:01.196592+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:50.045796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075167799999999998,"stat_sum":{"num_bytes":10260120,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":22,"num_read":824,"num_read_kb":6642,"num_write":109,"num_write_kb":5099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"56'171","reported_seq":1615,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.053917+0000","last_change":"2024-09-16T11:43:33.772949+0000","last_active":"2024-09-16T11:44:05.053917+0000","last_peered":"2024-09-16T11:44:05.053917+0000","last_clean":"2024-09-16T11:44:05.053917+0000","last_became_active":"2024-09-16T11:43:31.773661+0000","last_became_peered":"2024-09-16T11:43:31.773661+0000","last_unstale":"2024-09-16T11:44:05.053917+0000","last_undegraded":"2024-09-16T11:44:05.053917+0000","last_fullsized":"2024-09-16T11:44:05.053917+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:36.062796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023334699999999999,"stat_sum":{"num_bytes":20109099,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":17,"num_read":1606,"num_read_kb":10992,"num_write":176,"num_write_kb":4687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"56'242","reported_seq":1927,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.124164+0000","last_change":"2024-09-16T11:43:33.773384+0000","last_active":"2024-09-16T11:44:05.124164+0000","last_peered":"2024-09-16T11:44:05.124164+0000","last_clean":"2024-09-16T11:44:05.124164+0000","last_became_active":"2024-09-16T11:43:31.774523+0000","last_became_peered":"2024-09-16T11:43:31.774523+0000","last_unstale":"2024-09-16T11:44:05.124164+0000","last_undegraded":"2024-09-16T11:44:05.124164+0000","last_fullsized":"2024-09-16T11:44:05.124164+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:15.703792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038485,"stat_sum":{"num_bytes":25854084,"num_objects":42,"num_object_clones":3,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":29,"num_read":2682,"num_read_kb":12844,"num_write":205,"num_write_kb":2642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"56'410","reported_seq":2113,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.531895+0000","last_change":"2024-09-16T11:43:33.774141+0000","last_active":"2024-09-16T11:44:05.531895+0000","last_peered":"2024-09-16T11:44:05.531895+0000","last_clean":"2024-09-16T11:44:05.531895+0000","last_became_active":"2024-09-16T11:43:31.774794+0000","last_became_peered":"2024-09-16T11:43:31.774794+0000","last_unstale":"2024-09-16T11:44:05.531895+0000","last_undegraded":"2024-09-16T11:44:05.531895+0000","last_fullsized":"2024-09-16T11:44:05.531895+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:06:57.981702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010218040000000001,"stat_sum":{"num_bytes":21270251,"num_objects":57,"num_object_clones":8,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":35,"num_read":2917,"num_read_kb":6356,"num_write":543,"num_write_kb":6434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"53'507","reported_seq":787,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:02.399889+0000","last_change":"2024-09-16T11:43:33.777212+0000","last_active":"2024-09-16T11:44:02.399889+0000","last_peered":"2024-09-16T11:44:02.399889+0000","last_clean":"2024-09-16T11:44:02.399889+0000","last_became_active":"2024-09-16T11:43:31.769719+0000","last_became_peered":"2024-09-16T11:43:31.769719+0000","last_unstale":"2024-09-16T11:44:02.399889+0000","last_undegraded":"2024-09-16T11:44:02.399889+0000","last_fullsized":"2024-09-16T11:44:02.399889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:06.911046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035706099999999999,"stat_sum":{"num_bytes":17050010,"num_objects":28,"num_object_clones":1,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":17,"num_read":2075,"num_read_kb":2935,"num_write":731,"num_write_kb":1417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"56'118","reported_seq":549,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.525400+0000","last_change":"2024-09-16T11:43:33.774063+0000","last_active":"2024-09-16T11:44:05.525400+0000","last_peered":"2024-09-16T11:44:05.525400+0000","last_clean":"2024-09-16T11:44:05.525400+0000","last_became_active":"2024-09-16T11:43:31.776951+0000","last_became_peered":"2024-09-16T11:43:31.776951+0000","last_unstale":"2024-09-16T11:44:05.525400+0000","last_undegraded":"2024-09-16T11:44:05.525400+0000","last_fullsized":"2024-09-16T11:44:05.525400+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:28:39.026621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094366700000000003,"stat_sum":{"num_bytes":23235798,"num_objects":33,"num_object_clones":3,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":22,"num_read":439,"num_read_kb":1766,"num_write":101,"num_write_kb":5285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"53'197","reported_seq":1214,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:01.641410+0000","last_change":"2024-09-16T11:43:33.777913+0000","last_active":"2024-09-16T11:44:01.641410+0000","last_peered":"2024-09-16T11:44:01.641410+0000","last_clean":"2024-09-16T11:44:01.641410+0000","last_became_active":"2024-09-16T11:43:31.889807+0000","last_became_peered":"2024-09-16T11:43:31.889807+0000","last_unstale":"2024-09-16T11:44:01.641410+0000","last_undegraded":"2024-09-16T11:44:01.641410+0000","last_fullsized":"2024-09-16T11:44:01.641410+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:30.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071799400000000001,"stat_sum":{"num_bytes":18414640,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":25,"num_read":2431,"num_read_kb":6414,"num_write":167,"num_write_kb":4675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"55'166","reported_seq":832,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T11:44:04.328244+0000","last_change":"2024-09-16T11:43:34.248043+0000","last_active":"2024-09-16T11:44:04.328244+0000","last_peered":"2024-09-16T11:44:04.328244+0000","last_clean":"2024-09-16T11:44:04.328244+0000","last_became_active":"2024-09-16T11:43:31.776111+0000","last_became_peered":"2024-09-16T11:43:31.776111+0000","last_unstale":"2024-09-16T11:44:04.328244+0000","last_undegraded":"2024-09-16T11:44:04.328244+0000","last_fullsized":"2024-09-16T11:44:04.328244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:21:28.658052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024883199999999999,"stat_sum":{"num_bytes":16164004,"num_objects":35,"num_object_clones":2,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":28,"num_read":670,"num_read_kb":4129,"num_write":145,"num_write_kb":3253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"53'169","reported_seq":795,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:02.671953+0000","last_change":"2024-09-16T11:43:33.785207+0000","last_active":"2024-09-16T11:44:02.671953+0000","last_peered":"2024-09-16T11:44:02.671953+0000","last_clean":"2024-09-16T11:44:02.671953+0000","last_became_active":"2024-09-16T11:43:31.889629+0000","last_became_peered":"2024-09-16T11:43:31.889629+0000","last_unstale":"2024-09-16T11:44:02.671953+0000","last_undegraded":"2024-09-16T11:44:02.671953+0000","last_fullsized":"2024-09-16T11:44:02.671953+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:53.124562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022324199999999999,"stat_sum":{"num_bytes":23574665,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":31,"num_read":683,"num_read_kb":7246,"num_write":153,"num_write_kb":11725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"53'171","reported_seq":397,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:02.486447+0000","last_change":"2024-09-16T11:43:33.778129+0000","last_active":"2024-09-16T11:44:02.486447+0000","last_peered":"2024-09-16T11:44:02.486447+0000","last_clean":"2024-09-16T11:44:02.486447+0000","last_became_active":"2024-09-16T11:43:31.773947+0000","last_became_peered":"2024-09-16T11:43:31.773947+0000","last_unstale":"2024-09-16T11:44:02.486447+0000","last_undegraded":"2024-09-16T11:44:02.486447+0000","last_fullsized":"2024-09-16T11:44:02.486447+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:36:48.418141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085420199999999998,"stat_sum":{"num_bytes":19586605,"num_objects":32,"num_object_clones":2,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":23,"num_read":238,"num_read_kb":2158,"num_write":148,"num_write_kb":4348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"56'363","reported_seq":1479,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.250356+0000","last_change":"2024-09-16T11:43:33.773601+0000","last_active":"2024-09-16T11:44:05.250356+0000","last_peered":"2024-09-16T11:44:05.250356+0000","last_clean":"2024-09-16T11:44:05.250356+0000","last_became_active":"2024-09-16T11:43:31.776781+0000","last_became_peered":"2024-09-16T11:43:31.776781+0000","last_unstale":"2024-09-16T11:44:05.250356+0000","last_undegraded":"2024-09-16T11:44:05.250356+0000","last_fullsized":"2024-09-16T11:44:05.250356+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:14.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056674299999999998,"stat_sum":{"num_bytes":29338368,"num_objects":44,"num_object_clones":6,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":27,"num_read":2895,"num_read_kb":8798,"num_write":429,"num_write_kb":7746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"53'128","reported_seq":1185,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:02.333497+0000","last_change":"2024-09-16T11:43:33.773531+0000","last_active":"2024-09-16T11:44:02.333497+0000","last_peered":"2024-09-16T11:44:02.333497+0000","last_clean":"2024-09-16T11:44:02.333497+0000","last_became_active":"2024-09-16T11:43:31.780164+0000","last_became_peered":"2024-09-16T11:43:31.780164+0000","last_unstale":"2024-09-16T11:44:02.333497+0000","last_undegraded":"2024-09-16T11:44:02.333497+0000","last_fullsized":"2024-09-16T11:44:02.333497+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:56.846891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033807800000000001,"stat_sum":{"num_bytes":10717024,"num_objects":20,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":16,"num_read":1074,"num_read_kb":4952,"num_write":114,"num_write_kb":1107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"53'517","reported_seq":913,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:02.668833+0000","last_change":"2024-09-16T11:43:33.777531+0000","last_active":"2024-09-16T11:44:02.668833+0000","last_peered":"2024-09-16T11:44:02.668833+0000","last_clean":"2024-09-16T11:44:02.668833+0000","last_became_active":"2024-09-16T11:43:31.889754+0000","last_became_peered":"2024-09-16T11:43:31.889754+0000","last_unstale":"2024-09-16T11:44:02.668833+0000","last_undegraded":"2024-09-16T11:44:02.668833+0000","last_fullsized":"2024-09-16T11:44:02.668833+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:54.752770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062624299999999996,"stat_sum":{"num_bytes":10081277,"num_objects":47,"num_object_clones":2,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":38,"num_read":2329,"num_read_kb":6491,"num_write":759,"num_write_kb":5367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"53'205","reported_seq":1200,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:01.641334+0000","last_change":"2024-09-16T11:43:33.777407+0000","last_active":"2024-09-16T11:44:01.641334+0000","last_peered":"2024-09-16T11:44:01.641334+0000","last_clean":"2024-09-16T11:44:01.641334+0000","last_became_active":"2024-09-16T11:43:31.775962+0000","last_became_peered":"2024-09-16T11:43:31.775962+0000","last_unstale":"2024-09-16T11:44:01.641334+0000","last_undegraded":"2024-09-16T11:44:01.641334+0000","last_fullsized":"2024-09-16T11:44:01.641334+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:46.492489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062432399999999999,"stat_sum":{"num_bytes":5950530,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":29,"num_read":2410,"num_read_kb":7448,"num_write":191,"num_write_kb":1183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T11:44:03.892518+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:44:03.892518+0000","last_peered":"2024-09-16T11:44:03.892518+0000","last_clean":"2024-09-16T11:44:03.892518+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:44:03.892518+0000","last_undegraded":"2024-09-16T11:44:03.892518+0000","last_fullsized":"2024-09-16T11:44:03.892518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"55'224","reported_seq":1034,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.249917+0000","last_change":"2024-09-16T11:43:33.773359+0000","last_active":"2024-09-16T11:44:05.249917+0000","last_peered":"2024-09-16T11:44:05.249917+0000","last_clean":"2024-09-16T11:44:05.249917+0000","last_became_active":"2024-09-16T11:43:31.777591+0000","last_became_peered":"2024-09-16T11:43:31.777591+0000","last_unstale":"2024-09-16T11:44:05.249917+0000","last_undegraded":"2024-09-16T11:44:05.249917+0000","last_fullsized":"2024-09-16T11:44:05.249917+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:00:39.586913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037893300000000003,"stat_sum":{"num_bytes":12271664,"num_objects":30,"num_object_clones":3,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":20,"num_read":1907,"num_read_kb":4828,"num_write":195,"num_write_kb":4431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T11:43:59.951832+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:43:59.951832+0000","last_peered":"2024-09-16T11:43:59.951832+0000","last_clean":"2024-09-16T11:43:59.951832+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:43:59.951832+0000","last_undegraded":"2024-09-16T11:43:59.951832+0000","last_fullsized":"2024-09-16T11:43:59.951832+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"53'180","reported_seq":2144,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:02.417709+0000","last_change":"2024-09-16T11:43:33.777644+0000","last_active":"2024-09-16T11:44:02.417709+0000","last_peered":"2024-09-16T11:44:02.417709+0000","last_clean":"2024-09-16T11:44:02.417709+0000","last_became_active":"2024-09-16T11:43:31.892159+0000","last_became_peered":"2024-09-16T11:43:31.892159+0000","last_unstale":"2024-09-16T11:44:02.417709+0000","last_undegraded":"2024-09-16T11:44:02.417709+0000","last_fullsized":"2024-09-16T11:44:02.417709+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:42:14.042476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062271400000000001,"stat_sum":{"num_bytes":19657228,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":24,"num_read":2681,"num_read_kb":11780,"num_write":157,"num_write_kb":3812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T11:43:59.962031+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:43:59.962031+0000","last_peered":"2024-09-16T11:43:59.962031+0000","last_clean":"2024-09-16T11:43:59.962031+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:43:59.962031+0000","last_undegraded":"2024-09-16T11:43:59.962031+0000","last_fullsized":"2024-09-16T11:43:59.962031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"56'214","reported_seq":374,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:04.986959+0000","last_change":"2024-09-16T11:43:34.247771+0000","last_active":"2024-09-16T11:44:04.986959+0000","last_peered":"2024-09-16T11:44:04.986959+0000","last_clean":"2024-09-16T11:44:04.986959+0000","last_became_active":"2024-09-16T11:43:31.777896+0000","last_became_peered":"2024-09-16T11:43:31.777896+0000","last_unstale":"2024-09-16T11:44:04.986959+0000","last_undegraded":"2024-09-16T11:44:04.986959+0000","last_fullsized":"2024-09-16T11:44:04.986959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:35.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026844500000000002,"stat_sum":{"num_bytes":22717332,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":22,"num_read":202,"num_read_kb":2519,"num_write":197,"num_write_kb":2524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T11:43:59.951919+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:43:59.951919+0000","last_peered":"2024-09-16T11:43:59.951919+0000","last_clean":"2024-09-16T11:43:59.951919+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:43:59.951919+0000","last_undegraded":"2024-09-16T11:43:59.951919+0000","last_fullsized":"2024-09-16T11:43:59.951919+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"53'275","reported_seq":513,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:02.417659+0000","last_change":"2024-09-16T11:43:33.774367+0000","last_active":"2024-09-16T11:44:02.417659+0000","last_peered":"2024-09-16T11:44:02.417659+0000","last_clean":"2024-09-16T11:44:02.417659+0000","last_became_active":"2024-09-16T11:43:31.778506+0000","last_became_peered":"2024-09-16T11:43:31.778506+0000","last_unstale":"2024-09-16T11:44:02.417659+0000","last_undegraded":"2024-09-16T11:44:02.417659+0000","last_fullsized":"2024-09-16T11:44:02.417659+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:12.232493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010404660000000001,"stat_sum":{"num_bytes":1775702,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":16,"num_read":1162,"num_read_kb":2145,"num_write":408,"num_write_kb":1100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T11:44:03.890816+0000","last_change":"2024-09-16T11:43:22.438212+0000","last_active":"2024-09-16T11:44:03.890816+0000","last_peered":"2024-09-16T11:44:03.890816+0000","last_clean":"2024-09-16T11:44:03.890816+0000","last_became_active":"2024-09-16T11:43:22.437868+0000","last_became_peered":"2024-09-16T11:43:22.437868+0000","last_unstale":"2024-09-16T11:44:03.890816+0000","last_undegraded":"2024-09-16T11:44:03.890816+0000","last_fullsized":"2024-09-16T11:44:03.890816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:35.498933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":49,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T11:43:59.960322+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:43:59.960322+0000","last_peered":"2024-09-16T11:43:59.960322+0000","last_clean":"2024-09-16T11:43:59.960322+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:43:59.960322+0000","last_undegraded":"2024-09-16T11:43:59.960322+0000","last_fullsized":"2024-09-16T11:43:59.960322+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"53'250","reported_seq":820,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:02.647144+0000","last_change":"2024-09-16T11:43:33.774349+0000","last_active":"2024-09-16T11:44:02.647144+0000","last_peered":"2024-09-16T11:44:02.647144+0000","last_clean":"2024-09-16T11:44:02.647144+0000","last_became_active":"2024-09-16T11:43:31.777796+0000","last_became_peered":"2024-09-16T11:43:31.777796+0000","last_unstale":"2024-09-16T11:44:02.647144+0000","last_undegraded":"2024-09-16T11:44:02.647144+0000","last_fullsized":"2024-09-16T11:44:02.647144+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:33.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033368800000000001,"stat_sum":{"num_bytes":8441631,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":20,"num_read":1500,"num_read_kb":1602,"num_write":237,"num_write_kb":3226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T11:43:59.951586+0000","last_change":"2024-09-16T11:43:22.440136+0000","last_active":"2024-09-16T11:43:59.951586+0000","last_peered":"2024-09-16T11:43:59.951586+0000","last_clean":"2024-09-16T11:43:59.951586+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:43:59.951586+0000","last_undegraded":"2024-09-16T11:43:59.951586+0000","last_fullsized":"2024-09-16T11:43:59.951586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:18.461948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T11:44:03.887772+0000","last_change":"2024-09-16T11:43:16.732834+0000","last_active":"2024-09-16T11:44:03.887772+0000","last_peered":"2024-09-16T11:44:03.887772+0000","last_clean":"2024-09-16T11:44:03.887772+0000","last_became_active":"2024-09-16T11:43:14.724728+0000","last_became_peered":"2024-09-16T11:43:14.724728+0000","last_unstale":"2024-09-16T11:44:03.887772+0000","last_undegraded":"2024-09-16T11:44:03.887772+0000","last_fullsized":"2024-09-16T11:44:03.887772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:47.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"53'338","reported_seq":2505,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:02.379388+0000","last_change":"2024-09-16T11:43:33.774126+0000","last_active":"2024-09-16T11:44:02.379388+0000","last_peered":"2024-09-16T11:44:02.379388+0000","last_clean":"2024-09-16T11:44:02.379388+0000","last_became_active":"2024-09-16T11:43:31.777967+0000","last_became_peered":"2024-09-16T11:43:31.777967+0000","last_unstale":"2024-09-16T11:44:02.379388+0000","last_undegraded":"2024-09-16T11:44:02.379388+0000","last_fullsized":"2024-09-16T11:44:02.379388+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:43.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000373488,"stat_sum":{"num_bytes":49849587,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":29,"num_read":2977,"num_read_kb":14052,"num_write":434,"num_write_kb":4106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"56'1","reported_seq":43,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:04.897759+0000","last_change":"2024-09-16T11:43:22.437965+0000","last_active":"2024-09-16T11:44:04.897759+0000","last_peered":"2024-09-16T11:44:04.897759+0000","last_clean":"2024-09-16T11:44:04.897759+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:44:04.897759+0000","last_undegraded":"2024-09-16T11:44:04.897759+0000","last_fullsized":"2024-09-16T11:44:04.897759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:52.884360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T11:43:59.961984+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:43:59.961984+0000","last_peered":"2024-09-16T11:43:59.961984+0000","last_clean":"2024-09-16T11:43:59.961984+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:43:59.961984+0000","last_undegraded":"2024-09-16T11:43:59.961984+0000","last_fullsized":"2024-09-16T11:43:59.961984+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"56'183","reported_seq":555,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.396712+0000","last_change":"2024-09-16T11:43:34.248254+0000","last_active":"2024-09-16T11:44:05.396712+0000","last_peered":"2024-09-16T11:44:05.396712+0000","last_clean":"2024-09-16T11:44:05.396712+0000","last_became_active":"2024-09-16T11:43:31.776645+0000","last_became_peered":"2024-09-16T11:43:31.776645+0000","last_unstale":"2024-09-16T11:44:05.396712+0000","last_undegraded":"2024-09-16T11:44:05.396712+0000","last_fullsized":"2024-09-16T11:44:05.396712+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:53.198323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038670399999999998,"stat_sum":{"num_bytes":44997031,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":26,"num_read":398,"num_read_kb":3194,"num_write":155,"num_write_kb":2695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T11:43:59.874212+0000","last_change":"2024-09-16T11:43:22.587076+0000","last_active":"2024-09-16T11:43:59.874212+0000","last_peered":"2024-09-16T11:43:59.874212+0000","last_clean":"2024-09-16T11:43:59.874212+0000","last_became_active":"2024-09-16T11:43:22.586861+0000","last_became_peered":"2024-09-16T11:43:22.586861+0000","last_unstale":"2024-09-16T11:43:59.874212+0000","last_undegraded":"2024-09-16T11:43:59.874212+0000","last_fullsized":"2024-09-16T11:43:59.874212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:54:46.029558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":48,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T11:43:59.960282+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:43:59.960282+0000","last_peered":"2024-09-16T11:43:59.960282+0000","last_clean":"2024-09-16T11:43:59.960282+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:43:59.960282+0000","last_undegraded":"2024-09-16T11:43:59.960282+0000","last_fullsized":"2024-09-16T11:43:59.960282+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":134,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-16T11:43:59.961968+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:43:59.961968+0000","last_peered":"2024-09-16T11:43:59.961968+0000","last_clean":"2024-09-16T11:43:59.961968+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:43:59.961968+0000","last_undegraded":"2024-09-16T11:43:59.961968+0000","last_fullsized":"2024-09-16T11:43:59.961968+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"56'362","reported_seq":1994,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.400034+0000","last_change":"2024-09-16T11:43:33.781611+0000","last_active":"2024-09-16T11:44:05.400034+0000","last_peered":"2024-09-16T11:44:05.400034+0000","last_clean":"2024-09-16T11:44:05.400034+0000","last_became_active":"2024-09-16T11:43:31.782071+0000","last_became_peered":"2024-09-16T11:43:31.782071+0000","last_unstale":"2024-09-16T11:44:05.400034+0000","last_undegraded":"2024-09-16T11:44:05.400034+0000","last_fullsized":"2024-09-16T11:44:05.400034+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:15.516215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033637000000000001,"stat_sum":{"num_bytes":61180058,"num_objects":48,"num_object_clones":5,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":24,"num_read":2558,"num_read_kb":13125,"num_write":415,"num_write_kb":6333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"53'450","reported_seq":2047,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:02.610839+0000","last_change":"2024-09-16T11:43:33.785095+0000","last_active":"2024-09-16T11:44:02.610839+0000","last_peered":"2024-09-16T11:44:02.610839+0000","last_clean":"2024-09-16T11:44:02.610839+0000","last_became_active":"2024-09-16T11:43:31.892119+0000","last_became_peered":"2024-09-16T11:43:31.892119+0000","last_unstale":"2024-09-16T11:44:02.610839+0000","last_undegraded":"2024-09-16T11:44:02.610839+0000","last_fullsized":"2024-09-16T11:44:02.610839+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:04.609697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398253,"stat_sum":{"num_bytes":27453386,"num_objects":48,"num_object_clones":4,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":33,"num_read":2981,"num_read_kb":13434,"num_write":508,"num_write_kb":10533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"56'440","reported_seq":2198,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.090582+0000","last_change":"2024-09-16T11:43:33.773950+0000","last_active":"2024-09-16T11:44:05.090582+0000","last_peered":"2024-09-16T11:44:05.090582+0000","last_clean":"2024-09-16T11:44:05.090582+0000","last_became_active":"2024-09-16T11:43:31.775309+0000","last_became_peered":"2024-09-16T11:43:31.775309+0000","last_unstale":"2024-09-16T11:44:05.090582+0000","last_undegraded":"2024-09-16T11:44:05.090582+0000","last_fullsized":"2024-09-16T11:44:05.090582+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:23.106465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086187000000000004,"stat_sum":{"num_bytes":30413809,"num_objects":44,"num_object_clones":4,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":28,"num_read":4846,"num_read_kb":5296,"num_write":413,"num_write_kb":4246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"56'197","reported_seq":613,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.348233+0000","last_change":"2024-09-16T11:43:33.781852+0000","last_active":"2024-09-16T11:44:05.348233+0000","last_peered":"2024-09-16T11:44:05.348233+0000","last_clean":"2024-09-16T11:44:05.348233+0000","last_became_active":"2024-09-16T11:43:31.779384+0000","last_became_peered":"2024-09-16T11:43:31.779384+0000","last_unstale":"2024-09-16T11:44:05.348233+0000","last_undegraded":"2024-09-16T11:44:05.348233+0000","last_fullsized":"2024-09-16T11:44:05.348233+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:34.933146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029212900000000002,"stat_sum":{"num_bytes":14744044,"num_objects":44,"num_object_clones":4,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":34,"num_read":463,"num_read_kb":6784,"num_write":168,"num_write_kb":4854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"56'261","reported_seq":1053,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.540214+0000","last_change":"2024-09-16T11:43:34.247758+0000","last_active":"2024-09-16T11:44:05.540214+0000","last_peered":"2024-09-16T11:44:05.540214+0000","last_clean":"2024-09-16T11:44:05.540214+0000","last_became_active":"2024-09-16T11:43:31.775335+0000","last_became_peered":"2024-09-16T11:43:31.775335+0000","last_unstale":"2024-09-16T11:44:05.540214+0000","last_undegraded":"2024-09-16T11:44:05.540214+0000","last_fullsized":"2024-09-16T11:44:05.540214+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:07.521713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026563699999999999,"stat_sum":{"num_bytes":14043796,"num_objects":39,"num_object_clones":4,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":24,"num_read":1928,"num_read_kb":3047,"num_write":314,"num_write_kb":6769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"56'280","reported_seq":1241,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.544602+0000","last_change":"2024-09-16T11:43:33.773570+0000","last_active":"2024-09-16T11:44:05.544602+0000","last_peered":"2024-09-16T11:44:05.544602+0000","last_clean":"2024-09-16T11:44:05.544602+0000","last_became_active":"2024-09-16T11:43:31.774601+0000","last_became_peered":"2024-09-16T11:43:31.774601+0000","last_unstale":"2024-09-16T11:44:05.544602+0000","last_undegraded":"2024-09-16T11:44:05.544602+0000","last_fullsized":"2024-09-16T11:44:05.544602+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:12.584420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055828300000000002,"stat_sum":{"num_bytes":8945491,"num_objects":35,"num_object_clones":3,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":25,"num_read":2183,"num_read_kb":2151,"num_write":262,"num_write_kb":1300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"56'504","reported_seq":1145,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.399558+0000","last_change":"2024-09-16T11:43:33.773852+0000","last_active":"2024-09-16T11:44:05.399558+0000","last_peered":"2024-09-16T11:44:05.399558+0000","last_clean":"2024-09-16T11:44:05.399558+0000","last_became_active":"2024-09-16T11:43:31.772506+0000","last_became_peered":"2024-09-16T11:43:31.772506+0000","last_unstale":"2024-09-16T11:44:05.399558+0000","last_undegraded":"2024-09-16T11:44:05.399558+0000","last_fullsized":"2024-09-16T11:44:05.399558+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:36.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076283899999999997,"stat_sum":{"num_bytes":21979148,"num_objects":38,"num_object_clones":4,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":25,"num_read":2703,"num_read_kb":9956,"num_write":714,"num_write_kb":9690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"56'373","reported_seq":1021,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T11:44:05.465565+0000","last_change":"2024-09-16T11:43:33.773770+0000","last_active":"2024-09-16T11:44:05.465565+0000","last_peered":"2024-09-16T11:44:05.465565+0000","last_clean":"2024-09-16T11:44:05.465565+0000","last_became_active":"2024-09-16T11:43:31.779613+0000","last_became_peered":"2024-09-16T11:43:31.779613+0000","last_unstale":"2024-09-16T11:44:05.465565+0000","last_undegraded":"2024-09-16T11:44:05.465565+0000","last_fullsized":"2024-09-16T11:44:05.465565+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:40.995663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000720899,"stat_sum":{"num_bytes":15088606,"num_objects":44,"num_object_clones":6,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":27,"num_read":1716,"num_read_kb":4845,"num_write":442,"num_write_kb":1986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"53'135","reported_seq":489,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T11:44:02.431371+0000","last_change":"2024-09-16T11:43:33.773914+0000","last_active":"2024-09-16T11:44:02.431371+0000","last_peered":"2024-09-16T11:44:02.431371+0000","last_clean":"2024-09-16T11:44:02.431371+0000","last_became_active":"2024-09-16T11:43:31.776132+0000","last_became_peered":"2024-09-16T11:43:31.776132+0000","last_unstale":"2024-09-16T11:44:02.431371+0000","last_undegraded":"2024-09-16T11:44:02.431371+0000","last_fullsized":"2024-09-16T11:44:02.431371+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:53.489198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023678600000000001,"stat_sum":{"num_bytes":1951413,"num_objects":27,"num_object_clones":1,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":22,"num_read":713,"num_read_kb":1107,"num_write":115,"num_write_kb":1473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":634225871,"num_objects":1181,"num_object_clones":120,"num_object_copies":2362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1181,"num_whiteouts":796,"num_read":57157,"num_read_kb":198081,"num_write":9424,"num_write_kb":140227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15200256,"data_stored":120709672,"data_compressed":4327808,"data_compressed_allocated":11546624,"data_compressed_original":118783190,"omap_allocated":0,"internal_metadata":0},"log_size":8486,"ondisk_log_size":8486,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308872,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062968,"statfs":{"total":96636764160,"available":96320479232,"internally_reserved":0,"allocated":3276800,"data_stored":27604503,"data_compressed":634046,"data_compressed_allocated":2260992,"data_compressed_original":27109216,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7269999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308524,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063316,"statfs":{"total":96636764160,"available":96320835584,"internally_reserved":0,"allocated":2920448,"data_stored":18823760,"data_compressed":699834,"data_compressed_allocated":1810432,"data_compressed_original":18279437,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299868,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071972,"statfs":{"total":96636764160,"available":96329699328,"internally_reserved":0,"allocated":2527232,"data_stored":14634733,"data_compressed":546383,"data_compressed_allocated":1458176,"data_compressed_original":14122423,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299948,"kb_used_data":2596,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071892,"statfs":{"total":96636764160,"available":96329617408,"internally_reserved":0,"allocated":2658304,"data_stored":13251128,"data_compressed":605152,"data_compressed_allocated":1466368,"data_compressed_original":12660750,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.215}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307480,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064360,"statfs":{"total":96636764160,"available":96321904640,"internally_reserved":0,"allocated":1847296,"data_stored":10973377,"data_compressed":315967,"data_compressed_allocated":856064,"data_compressed_original":10514343,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312672,"kb_used_data":2836,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059168,"statfs":{"total":96636764160,"available":96316588032,"internally_reserved":0,"allocated":2904064,"data_stored":17019285,"data_compressed":750818,"data_compressed_allocated":1589248,"data_compressed_original":16355793,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311264,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94060576,"statfs":{"total":96636764160,"available":96318029824,"internally_reserved":0,"allocated":1466368,"data_stored":5865110,"data_compressed":157494,"data_compressed_allocated":524288,"data_compressed_original":5403296,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307152,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064688,"statfs":{"total":96636764160,"available":96322240512,"internally_reserved":0,"allocated":1507328,"data_stored":5006501,"data_compressed":330292,"data_compressed_allocated":634880,"data_compressed_original":4567040,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.215}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.17799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":892928,"data_stored":4205881,"data_compressed":363528,"data_compressed_allocated":643072,"data_compressed_original":4063232,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1945600,"data_stored":11315395,"data_compressed":614620,"data_compressed_allocated":1384448,"data_compressed_original":11014745,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1175552,"data_stored":9397986,"data_compressed":245116,"data_compressed_allocated":847872,"data_compressed_original":9220640,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1925120,"data_stored":13570041,"data_compressed":556569,"data_compressed_allocated":1437696,"data_compressed_original":13318652,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2916352,"data_stored":24469476,"data_compressed":864481,"data_compressed_allocated":2179072,"data_compressed_original":24065473,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2207744,"data_stored":17880043,"data_compressed":666050,"data_compressed_allocated":1740800,"data_compressed_original":17638498,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1716224,"data_stored":16496431,"data_compressed":418299,"data_compressed_allocated":1314816,"data_compressed_original":16307095,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2420736,"data_stored":23374419,"data_compressed":599145,"data_compressed_allocated":1998848,"data_compressed_original":23154855,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:44:06.285 INFO:tasks.thrashosds.thrasher:Setting 4.8 to [3, 6, 5, 1] 2024-09-16T11:44:06.286 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.8', '3', '6', '5', '1'] 2024-09-16T11:44:06.286 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 4.8 3 6 5 1 2024-09-16T11:44:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:3778 read 0x7bb7fa0 thru 0x7bc37ef (0xb850 bytes) 2024-09-16T11:44:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:3782 read 0x18e1912 thru 0x18e2877 (0xf66 bytes) 2024-09-16T11:44:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:3783 read 0x8011621 thru 0x8020ff2 (0xf9d2 bytes) 2024-09-16T11:44:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:3785 punch from 0x3d5555c to 0x3d57e61, (0x2905 bytes) 2024-09-16T11:44:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:3787 read 0x18e3ef6 thru 0x18ea5f4 (0x66ff bytes) 2024-09-16T11:44:06.306 INFO:teuthology.orchestra.run.smithi070.stdout:3789 read 0x44ad1d7 thru 0x44af405 (0x222f bytes) 2024-09-16T11:44:06.306 INFO:teuthology.orchestra.run.smithi070.stdout:3790 punch from 0x78ce947 to 0x78daba0, (0xc259 bytes) 2024-09-16T11:44:06.307 INFO:teuthology.orchestra.run.smithi070.stdout:3791 read 0xb41ca4 thru 0xb513b6 (0xf713 bytes) 2024-09-16T11:44:06.307 INFO:teuthology.orchestra.run.smithi070.stdout:3792 writesame 0x6996a1 thru 0x6a3211 data_size 0x1(0x9b71 bytes) 2024-09-16T11:44:06.345 INFO:teuthology.orchestra.run.smithi070.stdout:3794 punch from 0x64b4133 to 0x64b45ef, (0x4bc bytes) 2024-09-16T11:44:06.345 INFO:teuthology.orchestra.run.smithi070.stdout:3795 read 0x534b62b thru 0x535ac3d (0xf613 bytes) 2024-09-16T11:44:06.353 INFO:teuthology.orchestra.run.smithi070.stdout:3796 writesame 0xb7d4b68 thru 0xb7dc2e9 data_size 0x3(0x7782 bytes) 2024-09-16T11:44:06.353 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.353 INFO:teuthology.orchestra.run.smithi070.stdout:3798 read 0x7dc3f39 thru 0x7dd21ef (0xe2b7 bytes) 2024-09-16T11:44:06.353 INFO:teuthology.orchestra.run.smithi070.stdout:3799 punch from 0xb2a3533 to 0xb2abee6, (0x89b3 bytes) 2024-09-16T11:44:06.356 INFO:teuthology.orchestra.run.smithi070.stdout:3801 trunc from 0xb7dc2ea to 0x88db1aa 2024-09-16T11:44:06.371 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:06.369+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:06.380 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.380 INFO:teuthology.orchestra.run.smithi070.stdout:3803 read 0x2568a42 thru 0x257833e (0xf8fd bytes) 2024-09-16T11:44:06.388 INFO:teuthology.orchestra.run.smithi070.stdout:3804 write 0xa291b81 thru 0xa294c77 (0x30f7 bytes) 2024-09-16T11:44:06.393 INFO:teuthology.orchestra.run.smithi070.stdout:3805 read 0x14aa3b2 thru 0x14b8311 (0xdf60 bytes) 2024-09-16T11:44:06.393 INFO:teuthology.orchestra.run.smithi070.stdout:3807 write 0x48d9247 thru 0x48e3fa0 (0xad5a bytes) 2024-09-16T11:44:06.408 INFO:teuthology.orchestra.run.smithi070.stdout:3808 write 0x6e02e2b thru 0x6e03d21 (0xef7 bytes) 2024-09-16T11:44:06.411 INFO:teuthology.orchestra.run.smithi070.stdout:3809 write 0x222a56 thru 0x22ec57 (0xc202 bytes) 2024-09-16T11:44:06.415 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5357 4.b 4:d70b03a5:::rbd_data.11c18a2497a7.0000000000000026:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:44:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.982429+0000", 2024-09-16T11:44:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 24.428395932000001, 2024-09-16T11:44:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.018203652000000001, 2024-09-16T11:44:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5357 2024-09-16T11:44:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982429+0000", 2024-09-16T11:44:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982429+0000", 2024-09-16T11:44:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982431+0000", 2024-09-16T11:44:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.705e-06 2024-09-16T11:44:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982438+0000", 2024-09-16T11:44:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.588e-06 2024-09-16T11:44:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982454+0000", 2024-09-16T11:44:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5951e-05 2024-09-16T11:44:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982460+0000", 2024-09-16T11:44:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9329999999999998e-06 2024-09-16T11:44:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982515+0000", 2024-09-16T11:44:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4583000000000002e-05 2024-09-16T11:44:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982609+0000", 2024-09-16T11:44:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3857999999999995e-05 2024-09-16T11:44:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:44:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.985720+0000", 2024-09-16T11:44:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031111020000000001 2024-09-16T11:44:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000561+0000", 2024-09-16T11:44:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014841221999999999 2024-09-16T11:44:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000571+0000", 2024-09-16T11:44:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8039999999999996e-06 2024-09-16T11:44:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000606+0000", 2024-09-16T11:44:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5197999999999997e-05 2024-09-16T11:44:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000633+0000", 2024-09-16T11:44:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6707999999999999e-05 2024-09-16T11:44:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:9081 4.b 4:d317bc5b:::rbd_data.11c1dc795e15.000000000000000d:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:44:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:44:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.356869468999999, 2024-09-16T11:44:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027796406999999999, 2024-09-16T11:44:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 9081 2024-09-16T11:44:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053957+0000", 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7010000000000001e-06 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053965+0000", 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5939999999999996e-06 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053980+0000", 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5099000000000001e-05 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053986+0000", 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5759999999999996e-06 2024-09-16T11:44:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.054023+0000", 2024-09-16T11:44:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999999e-05 2024-09-16T11:44:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.054076+0000", 2024-09-16T11:44:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2852999999999997e-05 2024-09-16T11:44:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:44:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.058880+0000", 2024-09-16T11:44:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048042639999999999 2024-09-16T11:44:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081663+0000", 2024-09-16T11:44:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022782908000000001 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081672+0000", 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0440000000000001e-06 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081707+0000", 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4986e-05 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081752+0000", 2024-09-16T11:44:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4781999999999999e-05 2024-09-16T11:44:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.4545.0:9081 4.b e40/28 ondisk, result = 0)", 2024-09-16T11:44:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.073300+0000", 2024-09-16T11:44:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.337525126999999, 2024-09-16T11:44:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0083654630000000001, 2024-09-16T11:44:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073300+0000", 2024-09-16T11:44:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073300+0000", 2024-09-16T11:44:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073301+0000", 2024-09-16T11:44:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2100000000000001e-06 2024-09-16T11:44:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073307+0000", 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.071e-06 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073320+0000", 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3149000000000001e-05 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.073329+0000", 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4069999999999993e-06 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081650+0000", 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.008321541 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081658+0000", 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6260000000000003e-06 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081665+0000", 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4590000000000001e-06 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:9113 4.b 4:d317bc5b:::rbd_data.11c1dc795e15.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:44:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.119551+0000", 2024-09-16T11:44:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 20.291274009999999, 2024-09-16T11:44:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.020116319000000001, 2024-09-16T11:44:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 9113 2024-09-16T11:44:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119551+0000", 2024-09-16T11:44:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119551+0000", 2024-09-16T11:44:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119553+0000", 2024-09-16T11:44:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.782e-06 2024-09-16T11:44:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119563+0000", 2024-09-16T11:44:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0325e-05 2024-09-16T11:44:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119577+0000", 2024-09-16T11:44:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.382e-05 2024-09-16T11:44:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119583+0000", 2024-09-16T11:44:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.57e-06 2024-09-16T11:44:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119620+0000", 2024-09-16T11:44:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7737000000000001e-05 2024-09-16T11:44:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119673+0000", 2024-09-16T11:44:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3152000000000002e-05 2024-09-16T11:44:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:44:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.122945+0000", 2024-09-16T11:44:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0032719390000000002 2024-09-16T11:44:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139568+0000", 2024-09-16T11:44:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016622839 2024-09-16T11:44:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139606+0000", 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8006000000000002e-05 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139641+0000", 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5076000000000001e-05 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139667+0000", 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6072999999999999e-05 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32306 4.1e 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.501566+0000", 2024-09-16T11:44:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.9092588319999999, 2024-09-16T11:44:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093950460000000006, 2024-09-16T11:44:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32306 2024-09-16T11:44:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501566+0000", 2024-09-16T11:44:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501566+0000", 2024-09-16T11:44:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501569+0000", 2024-09-16T11:44:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3290000000000001e-06 2024-09-16T11:44:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501576+0000", 2024-09-16T11:44:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9829999999999999e-06 2024-09-16T11:44:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501595+0000", 2024-09-16T11:44:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9548e-05 2024-09-16T11:44:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501603+0000", 2024-09-16T11:44:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.729e-06 2024-09-16T11:44:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501664+0000", 2024-09-16T11:44:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1161000000000006e-05 2024-09-16T11:44:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501747+0000", 2024-09-16T11:44:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2888999999999999e-05 2024-09-16T11:44:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501854+0000", 2024-09-16T11:44:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010701600000000001 2024-09-16T11:44:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.502516+0000", 2024-09-16T11:44:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066228299999999995 2024-09-16T11:44:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.510891+0000", 2024-09-16T11:44:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0083744539999999999 2024-09-16T11:44:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.510942+0000", 2024-09-16T11:44:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1325000000000001e-05 2024-09-16T11:44:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.510961+0000", 2024-09-16T11:44:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9329000000000001e-05 2024-09-16T11:44:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32402 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515533+0000", 2024-09-16T11:44:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8952923469999998, 2024-09-16T11:44:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0083807889999999996, 2024-09-16T11:44:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32402 2024-09-16T11:44:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515533+0000", 2024-09-16T11:44:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515533+0000", 2024-09-16T11:44:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515533+0000", 2024-09-16T11:44:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:44:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515535+0000", 2024-09-16T11:44:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.618e-06 2024-09-16T11:44:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515540+0000", 2024-09-16T11:44:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2850000000000002e-06 2024-09-16T11:44:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515543+0000", 2024-09-16T11:44:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.315e-06 2024-09-16T11:44:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.523778+0000", 2024-09-16T11:44:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0082349520000000002 2024-09-16T11:44:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.523850+0000", 2024-09-16T11:44:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2069000000000004e-05 2024-09-16T11:44:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.523914+0000", 2024-09-16T11:44:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3038000000000002e-05 2024-09-16T11:44:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32403 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515547+0000", 2024-09-16T11:44:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8952779580000003, 2024-09-16T11:44:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0084589130000000002, 2024-09-16T11:44:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32403 2024-09-16T11:44:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515547+0000", 2024-09-16T11:44:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515547+0000", 2024-09-16T11:44:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515548+0000", 2024-09-16T11:44:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4000000000000002e-07 2024-09-16T11:44:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515549+0000", 2024-09-16T11:44:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.77e-06 2024-09-16T11:44:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515555+0000", 2024-09-16T11:44:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9850000000000003e-06 2024-09-16T11:44:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515559+0000", 2024-09-16T11:44:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3840000000000001e-06 2024-09-16T11:44:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.523936+0000", 2024-09-16T11:44:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0083773709999999998 2024-09-16T11:44:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.523973+0000", 2024-09-16T11:44:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6684999999999999e-05 2024-09-16T11:44:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524006+0000", 2024-09-16T11:44:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3278000000000002e-05 2024-09-16T11:44:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32404 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515562+0000", 2024-09-16T11:44:06.461 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8952626549999998, 2024-09-16T11:44:06.461 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0086125379999999994, 2024-09-16T11:44:06.461 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.461 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.461 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32404 2024-09-16T11:44:06.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.461 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515562+0000", 2024-09-16T11:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515562+0000", 2024-09-16T11:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515563+0000", 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515565+0000", 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.821e-06 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515570+0000", 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5650000000000001e-06 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515573+0000", 2024-09-16T11:44:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1990000000000002e-06 2024-09-16T11:44:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524034+0000", 2024-09-16T11:44:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0084602259999999995 2024-09-16T11:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524112+0000", 2024-09-16T11:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8802999999999998e-05 2024-09-16T11:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524175+0000", 2024-09-16T11:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2572999999999999e-05 2024-09-16T11:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32405 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515577+0000", 2024-09-16T11:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8952482919999998, 2024-09-16T11:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0086946499999999999, 2024-09-16T11:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32405 2024-09-16T11:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515577+0000", 2024-09-16T11:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515577+0000", 2024-09-16T11:44:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515577+0000", 2024-09-16T11:44:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3599999999999999e-07 2024-09-16T11:44:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515579+0000", 2024-09-16T11:44:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6640000000000001e-06 2024-09-16T11:44:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515584+0000", 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.293e-06 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515587+0000", 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1949999999999999e-06 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524199+0000", 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0086117180000000008 2024-09-16T11:44:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524238+0000", 2024-09-16T11:44:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8652999999999997e-05 2024-09-16T11:44:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524271+0000", 2024-09-16T11:44:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3791000000000002e-05 2024-09-16T11:44:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32406 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515591+0000", 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8952344859999997, 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0088408800000000006, 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32406 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515591+0000", 2024-09-16T11:44:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515591+0000", 2024-09-16T11:44:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515591+0000", 2024-09-16T11:44:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2300000000000002e-07 2024-09-16T11:44:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515593+0000", 2024-09-16T11:44:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6750000000000001e-06 2024-09-16T11:44:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515598+0000", 2024-09-16T11:44:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:44:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515601+0000", 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1820000000000002e-06 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524297+0000", 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0086959670000000006 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524369+0000", 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2045000000000006e-05 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524431+0000", 2024-09-16T11:44:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2210000000000002e-05 2024-09-16T11:44:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32407 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515605+0000", 2024-09-16T11:44:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.895220514, 2024-09-16T11:44:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0089202120000000003, 2024-09-16T11:44:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32407 2024-09-16T11:44:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515605+0000", 2024-09-16T11:44:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515605+0000", 2024-09-16T11:44:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515605+0000", 2024-09-16T11:44:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.27e-07 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515606+0000", 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.592e-06 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515612+0000", 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4500000000000003e-06 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515615+0000", 2024-09-16T11:44:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2909999999999999e-06 2024-09-16T11:44:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524454+0000", 2024-09-16T11:44:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0088386100000000002 2024-09-16T11:44:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524492+0000", 2024-09-16T11:44:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8383000000000002e-05 2024-09-16T11:44:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524525+0000", 2024-09-16T11:44:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2558999999999998e-05 2024-09-16T11:44:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32408 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515625+0000", 2024-09-16T11:44:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8951998149999998, 2024-09-16T11:44:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0090628719999999996, 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32408 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515625+0000", 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515625+0000", 2024-09-16T11:44:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515626+0000", 2024-09-16T11:44:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6499999999999999e-07 2024-09-16T11:44:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515632+0000", 2024-09-16T11:44:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3990000000000001e-06 2024-09-16T11:44:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515639+0000", 2024-09-16T11:44:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0360000000000001e-06 2024-09-16T11:44:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515643+0000", 2024-09-16T11:44:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6650000000000001e-06 2024-09-16T11:44:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524551+0000", 2024-09-16T11:44:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0089076520000000003 2024-09-16T11:44:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524624+0000", 2024-09-16T11:44:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3349999999999994e-05 2024-09-16T11:44:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524688+0000", 2024-09-16T11:44:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4204999999999999e-05 2024-09-16T11:44:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32409 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515647+0000", 2024-09-16T11:44:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8951783410000003, 2024-09-16T11:44:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0091245730000000004, 2024-09-16T11:44:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32409 2024-09-16T11:44:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515647+0000", 2024-09-16T11:44:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515647+0000", 2024-09-16T11:44:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515647+0000", 2024-09-16T11:44:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7100000000000002e-07 2024-09-16T11:44:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515649+0000", 2024-09-16T11:44:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7659999999999999e-06 2024-09-16T11:44:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515655+0000", 2024-09-16T11:44:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6080000000000003e-06 2024-09-16T11:44:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515658+0000", 2024-09-16T11:44:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2590000000000001e-06 2024-09-16T11:44:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524699+0000", 2024-09-16T11:44:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0090409919999999994 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524735+0000", 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6023999999999999e-05 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524771+0000", 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6452999999999998e-05 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32410 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515661+0000", 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.895163868, 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0092737480000000001, 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32410 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515661+0000", 2024-09-16T11:44:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515661+0000", 2024-09-16T11:44:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515662+0000", 2024-09-16T11:44:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.46e-07 2024-09-16T11:44:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515663+0000", 2024-09-16T11:44:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5519999999999999e-06 2024-09-16T11:44:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515668+0000", 2024-09-16T11:44:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3430000000000003e-06 2024-09-16T11:44:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515672+0000", 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2610000000000002e-06 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524798+0000", 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0091261139999999994 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524872+0000", 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4443999999999994e-05 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524935+0000", 2024-09-16T11:44:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2687999999999993e-05 2024-09-16T11:44:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32411 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515676+0000", 2024-09-16T11:44:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8951487450000002, 2024-09-16T11:44:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093416420000000007, 2024-09-16T11:44:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32411 2024-09-16T11:44:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515676+0000", 2024-09-16T11:44:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515676+0000", 2024-09-16T11:44:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515677+0000", 2024-09-16T11:44:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:44:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515678+0000", 2024-09-16T11:44:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6729999999999999e-06 2024-09-16T11:44:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515684+0000", 2024-09-16T11:44:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:44:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515687+0000", 2024-09-16T11:44:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2650000000000001e-06 2024-09-16T11:44:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524945+0000", 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0092578490000000003 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524982+0000", 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7392999999999999e-05 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525018+0000", 2024-09-16T11:44:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5645000000000002e-05 2024-09-16T11:44:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32412 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515691+0000", 2024-09-16T11:44:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8951344519999997, 2024-09-16T11:44:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0094970969999999995, 2024-09-16T11:44:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32412 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515691+0000", 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515691+0000", 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515691+0000", 2024-09-16T11:44:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3099999999999998e-07 2024-09-16T11:44:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515693+0000", 2024-09-16T11:44:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6610000000000001e-06 2024-09-16T11:44:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515698+0000", 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.592e-06 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515702+0000", 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.326e-06 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525046+0000", 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093440069999999997 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525125+0000", 2024-09-16T11:44:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9629000000000006e-05 2024-09-16T11:44:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525188+0000", 2024-09-16T11:44:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2351e-05 2024-09-16T11:44:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32413 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515705+0000", 2024-09-16T11:44:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8951197979999996, 2024-09-16T11:44:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0095604680000000008, 2024-09-16T11:44:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32413 2024-09-16T11:44:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515705+0000", 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515705+0000", 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515706+0000", 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2599999999999998e-07 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515707+0000", 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6440000000000001e-06 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515713+0000", 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7679999999999997e-06 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515716+0000", 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.349e-06 2024-09-16T11:44:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525198+0000", 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0094821130000000003 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525233+0000", 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4430000000000001e-05 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525266+0000", 2024-09-16T11:44:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2838e-05 2024-09-16T11:44:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32414 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515720+0000", 2024-09-16T11:44:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8951050269999996, 2024-09-16T11:44:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0097012409999999993, 2024-09-16T11:44:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32414 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515720+0000", 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515720+0000", 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515720+0000", 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515722+0000", 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8369999999999999e-06 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515728+0000", 2024-09-16T11:44:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.524e-06 2024-09-16T11:44:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515731+0000", 2024-09-16T11:44:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3060000000000002e-06 2024-09-16T11:44:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525284+0000", 2024-09-16T11:44:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0095529299999999994 2024-09-16T11:44:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525359+0000", 2024-09-16T11:44:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5087999999999997e-05 2024-09-16T11:44:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525421+0000", 2024-09-16T11:44:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2126999999999997e-05 2024-09-16T11:44:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32718 4.3 4:c5c936b1:::rbd_data.11c1cab081b3.0000000000000009:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.595459+0000", 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8153656209999998, 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0094432419999999993, 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32718 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595459+0000", 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595459+0000", 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595460+0000", 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.54e-07 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595464+0000", 2024-09-16T11:44:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1710000000000004e-06 2024-09-16T11:44:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595477+0000", 2024-09-16T11:44:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2197e-05 2024-09-16T11:44:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595482+0000", 2024-09-16T11:44:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0130000000000001e-06 2024-09-16T11:44:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595526+0000", 2024-09-16T11:44:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4363000000000002e-05 2024-09-16T11:44:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595609+0000", 2024-09-16T11:44:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2641999999999999e-05 2024-09-16T11:44:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:44:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.597861+0000", 2024-09-16T11:44:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022526059999999999 2024-09-16T11:44:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.604625+0000", 2024-09-16T11:44:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.006764117 2024-09-16T11:44:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.604839+0000", 2024-09-16T11:44:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000213663 2024-09-16T11:44:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.604869+0000", 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9688000000000001e-05 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.604903+0000", 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3927999999999998e-05 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:38387 4.6 4:60ec0988:::rbd_data.11c1ae8c7c6e.000000000000000b:18 [sparse-read 5111808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-16T11:44:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:04.960551+0000", 2024-09-16T11:44:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.4502738260000001, 2024-09-16T11:44:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0084680610000000007, 2024-09-16T11:44:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 38387 2024-09-16T11:44:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:04.960551+0000", 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:04.960551+0000", 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:04.960551+0000", 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0599999999999999e-07 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:04.960553+0000", 2024-09-16T11:44:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1009999999999999e-06 2024-09-16T11:44:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:04.960555+0000", 2024-09-16T11:44:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.813e-06 2024-09-16T11:44:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:04.960557+0000", 2024-09-16T11:44:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0099999999999998e-06 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:04.968832+0000", 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0082742890000000006 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:04.968901+0000", 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9663e-05 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:06.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:04.969019+0000", 2024-09-16T11:44:06.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000117979 2024-09-16T11:44:06.523 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.523 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.523 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.523 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.523 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.523 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:06.525 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:06.527 INFO:teuthology.orchestra.run.smithi070.stdout:3811 writesame 0xc53b8fd thru 0xc53e329 data_size 0x3(0x2a2d bytes) 2024-09-16T11:44:06.527 INFO:teuthology.orchestra.run.smithi070.stdout:3812 read 0x4788109 thru 0x478e300 (0x61f8 bytes) 2024-09-16T11:44:06.527 INFO:teuthology.orchestra.run.smithi070.stdout:3815 read 0x2049def thru 0x2057751 (0xd963 bytes) 2024-09-16T11:44:06.527 INFO:teuthology.orchestra.run.smithi070.stdout:3816 writesame 0x15beee4 thru 0x15c0939 data_size 0x2(0x1a56 bytes) 2024-09-16T11:44:06.527 INFO:teuthology.orchestra.run.smithi070.stdout:3817 read 0x6319cb4 thru 0x6324d26 (0xb073 bytes) 2024-09-16T11:44:06.527 INFO:teuthology.orchestra.run.smithi070.stdout:3818 read 0x62d8b83 thru 0x62e1291 (0x870f bytes) 2024-09-16T11:44:06.527 INFO:teuthology.orchestra.run.smithi070.stdout:3819 punch from 0x4fdfb27 to 0x4fe7648, (0x7b21 bytes) 2024-09-16T11:44:06.527 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.527 INFO:teuthology.orchestra.run.smithi070.stdout:3821 punch from 0x35c7b42 to 0x35d0452, (0x8910 bytes) 2024-09-16T11:44:06.527 INFO:teuthology.orchestra.run.smithi070.stdout:3822 read 0x14e71fb thru 0x14f6c3f (0xfa45 bytes) 2024-09-16T11:44:06.527 INFO:teuthology.orchestra.run.smithi070.stdout:3823 punch from 0x74daae5 to 0x74e6a90, (0xbfab bytes) 2024-09-16T11:44:06.528 INFO:teuthology.orchestra.run.smithi070.stdout:3824 write 0xd4137e5 thru 0xd4201b1 (0xc9cd bytes) 2024-09-16T11:44:06.528 INFO:teuthology.orchestra.run.smithi070.stdout:3826 punch from 0x5f6dc19 to 0x5f747a1, (0x6b88 bytes) 2024-09-16T11:44:06.528 INFO:teuthology.orchestra.run.smithi070.stdout:3827 read 0xca86e26 thru 0xca96ba6 (0xfd81 bytes) 2024-09-16T11:44:06.528 INFO:teuthology.orchestra.run.smithi070.stdout:3828 read 0x8bec34c thru 0x8bedb29 (0x17de bytes) 2024-09-16T11:44:06.528 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.528 INFO:teuthology.orchestra.run.smithi070.stdout:3831 read 0x96b646a thru 0x96c2881 (0xc418 bytes) 2024-09-16T11:44:06.528 INFO:teuthology.orchestra.run.smithi070.stdout:3832 read 0x68fe0ff thru 0x6902929 (0x482b bytes) 2024-09-16T11:44:06.528 INFO:teuthology.orchestra.run.smithi070.stdout:3834 read 0x3c43554 thru 0x3c498cf (0x637c bytes) 2024-09-16T11:44:06.528 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.528 INFO:teuthology.orchestra.run.smithi070.stdout:3838 punch from 0xc7c8ac to 0xc8c168, (0xf8bc bytes) 2024-09-16T11:44:06.528 INFO:teuthology.orchestra.run.smithi070.stdout:3839 write 0xe06c9a7 thru 0xe076cd1 (0xa32b bytes) 2024-09-16T11:44:06.528 INFO:teuthology.orchestra.run.smithi070.stdout:3840 write 0xe97a48f thru 0xe989c03 (0xf775 bytes) 2024-09-16T11:44:06.528 INFO:teuthology.orchestra.run.smithi070.stdout:3841 trunc from 0xe989c04 to 0x13dd2e4 2024-09-16T11:44:06.529 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:06.522+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:06.564 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.564 INFO:teuthology.orchestra.run.smithi070.stdout:3844 punch from 0x10bba55 to 0x10c48d1, (0x8e7c bytes) 2024-09-16T11:44:06.564 INFO:teuthology.orchestra.run.smithi070.stdout:3845 punch from 0xe227c7 to 0xe2b85b, (0x9094 bytes) 2024-09-16T11:44:06.564 INFO:teuthology.orchestra.run.smithi070.stdout:3846 read 0xe6a166 thru 0xe7514a (0xafe5 bytes) 2024-09-16T11:44:06.564 INFO:teuthology.orchestra.run.smithi070.stdout:3847 read 0xc440ea thru 0xc53eee (0xfe05 bytes) 2024-09-16T11:44:06.566 INFO:teuthology.orchestra.run.smithi070.stdout:3851 read 0x10e4bd thru 0x119583 (0xb0c7 bytes) 2024-09-16T11:44:06.567 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.567 INFO:teuthology.orchestra.run.smithi070.stdout:3854 trunc from 0x13dd2e4 to 0x954132 2024-09-16T11:44:06.612 INFO:teuthology.orchestra.run.smithi070.stdout:3855 read 0x171cf3 thru 0x179425 (0x7733 bytes) 2024-09-16T11:44:06.612 INFO:teuthology.orchestra.run.smithi070.stdout:3857 read 0x3a69ba thru 0x3b4423 (0xda6a bytes) 2024-09-16T11:44:06.612 INFO:teuthology.orchestra.run.smithi070.stdout:3859 read 0x5a992e thru 0x5b2484 (0x8b57 bytes) 2024-09-16T11:44:06.613 INFO:teuthology.orchestra.run.smithi070.stdout:3860 trunc from 0x954132 to 0xf82541 2024-09-16T11:44:06.617 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.625 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:06.623+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:06.629 INFO:teuthology.orchestra.run.smithi070.stdout:3863 write 0x6996251 thru 0x6998aa1 (0x2851 bytes) 2024-09-16T11:44:06.633 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.634 INFO:teuthology.orchestra.run.smithi070.stdout:3867 trunc from 0x6998aa2 to 0x5c1e23c 2024-09-16T11:44:06.643 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.643 INFO:teuthology.orchestra.run.smithi070.stdout:3869 read 0x50ca0e9 thru 0x50cad45 (0xc5d bytes) 2024-09-16T11:44:06.651 INFO:teuthology.orchestra.run.smithi070.stdout:3870 trunc from 0x5c1e23c to 0xbbff2cf 2024-09-16T11:44:06.656 INFO:teuthology.orchestra.run.smithi070.stdout:3871 write 0x1481f24 thru 0x1486bda (0x4cb7 bytes) 2024-09-16T11:44:06.661 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:06.661 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:06.661 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.661 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4545.0:43122 4.14 e57/28 4:2abb46d4:::rbd_object_map.11c1e170f5f0:head v 57'327, mlcod=57'322)", 2024-09-16T11:44:06.662 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:06.658807+0000", 2024-09-16T11:44:06.662 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001223324, 2024-09-16T11:44:06.662 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001244883, 2024-09-16T11:44:06.662 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.662 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:06.662 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.662 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.662 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.662 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.658807+0000", 2024-09-16T11:44:06.662 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.662 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.662 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.663 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.658807+0000", 2024-09-16T11:44:06.663 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.663 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.663 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.663 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.658808+0000", 2024-09-16T11:44:06.663 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7999999999999995e-07 2024-09-16T11:44:06.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.663 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.663 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.663 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.658813+0000", 2024-09-16T11:44:06.663 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2630000000000003e-06 2024-09-16T11:44:06.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.664 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.664 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.658823+0000", 2024-09-16T11:44:06.664 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0049e-05 2024-09-16T11:44:06.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.664 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.664 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.658828+0000", 2024-09-16T11:44:06.664 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4049999999999999e-06 2024-09-16T11:44:06.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.665 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.665 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.658862+0000", 2024-09-16T11:44:06.665 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3890999999999998e-05 2024-09-16T11:44:06.665 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.665 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.665 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:06.665 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.658880+0000", 2024-09-16T11:44:06.665 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7640999999999999e-05 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.4545.0:43124 4.10 e57/28)", 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:06.659893+0000", 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.000137477, 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022755400000000001, 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.666 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:06.667 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.659893+0000", 2024-09-16T11:44:06.667 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.667 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.667 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:06.667 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.659893+0000", 2024-09-16T11:44:06.667 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:06.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.667 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.667 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:06.667 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.659893+0000", 2024-09-16T11:44:06.667 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5099999999999999e-07 2024-09-16T11:44:06.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.668 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.668 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:06.668 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.659904+0000", 2024-09-16T11:44:06.668 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0855e-05 2024-09-16T11:44:06.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.668 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.668 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:06.668 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.659917+0000", 2024-09-16T11:44:06.668 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3095000000000001e-05 2024-09-16T11:44:06.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.659937+0000", 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9794000000000001e-05 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.659997+0000", 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9463000000000003e-05 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:06.669 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:06.670 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 2 2024-09-16T11:44:06.670 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:06.674 INFO:teuthology.orchestra.run.smithi070.stdout:3872 trunc from 0xbbff2cf to 0x1f71a8e 2024-09-16T11:44:06.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:06.681 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.698 INFO:teuthology.orchestra.run.smithi070.stdout:3874 write 0xd08ca3a thru 0xd090cdc (0x42a3 bytes) 2024-09-16T11:44:06.703 INFO:teuthology.orchestra.run.smithi070.stdout:3876 read 0x470183a thru 0x4710e9f (0xf666 bytes) 2024-09-16T11:44:06.703 INFO:teuthology.orchestra.run.smithi070.stdout:3877 punch from 0x1c869f9 to 0x1c8aba5, (0x41ac bytes) 2024-09-16T11:44:06.703 INFO:teuthology.orchestra.run.smithi070.stdout:3878 read 0x176ae62 thru 0x17763d4 (0xb573 bytes) 2024-09-16T11:44:06.703 INFO:teuthology.orchestra.run.smithi070.stdout:3879 read 0xc48c182 thru 0xc48f735 (0x35b4 bytes) 2024-09-16T11:44:06.703 INFO:teuthology.orchestra.run.smithi070.stdout:3880 write 0x57a3946 thru 0x57aa148 (0x6803 bytes) 2024-09-16T11:44:06.710 INFO:teuthology.orchestra.run.smithi070.stdout:3881 read 0x3b23ca1 thru 0x3b24042 (0x3a2 bytes) 2024-09-16T11:44:06.710 INFO:teuthology.orchestra.run.smithi070.stdout:3882 punch from 0xbf69c44 to 0xbf6f26f, (0x562b bytes) 2024-09-16T11:44:06.710 INFO:teuthology.orchestra.run.smithi070.stdout:3883 write 0x7552a45 thru 0x755eff8 (0xc5b4 bytes) 2024-09-16T11:44:06.718 INFO:teuthology.orchestra.run.smithi070.stdout:3887 writesame 0x8a5e58e thru 0x8a6d454 data_size 0x1(0xeec7 bytes) 2024-09-16T11:44:06.725 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:06.723+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:06.731 INFO:teuthology.orchestra.run.smithi070.stdout:3888 read 0x49b5b9 thru 0x4a2539 (0x6f81 bytes) 2024-09-16T11:44:06.732 INFO:teuthology.orchestra.run.smithi070.stdout:3889 write 0x2562c6 thru 0x2589ae (0x26e9 bytes) 2024-09-16T11:44:06.734 INFO:teuthology.orchestra.run.smithi070.stdout:3890 punch from 0x6dbe775 to 0x6dc78db, (0x9166 bytes) 2024-09-16T11:44:06.734 INFO:teuthology.orchestra.run.smithi070.stdout:3891 write 0x63f0a09 thru 0x63fc396 (0xb98e bytes) 2024-09-16T11:44:06.743 INFO:teuthology.orchestra.run.smithi070.stdout:3892 write 0x4432212 thru 0x443426b (0x205a bytes) 2024-09-16T11:44:06.751 INFO:teuthology.orchestra.run.smithi070.stdout:3893 writesame 0xd64a0ee thru 0xd650bc9 data_size 0xe(0x6adc bytes) 2024-09-16T11:44:06.751 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.751 INFO:teuthology.orchestra.run.smithi070.stdout:3895 write 0x7cda9dc thru 0x7cdac7e (0x2a3 bytes) 2024-09-16T11:44:06.754 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.754 INFO:teuthology.orchestra.run.smithi070.stdout:3898 punch from 0x9ccd8aa to 0x9cd9e6b, (0xc5c1 bytes) 2024-09-16T11:44:06.754 INFO:teuthology.orchestra.run.smithi070.stdout:3900 trunc from 0xd650bca to 0xdb679f0 2024-09-16T11:44:06.759 INFO:teuthology.orchestra.run.smithi070.stdout:3901 write 0x152a406 thru 0x152d02e (0x2c29 bytes) 2024-09-16T11:44:06.760 INFO:teuthology.orchestra.run.smithi070.stdout:3904 read 0x2dd8598 thru 0x2de6d1e (0xe787 bytes) 2024-09-16T11:44:06.760 INFO:teuthology.orchestra.run.smithi070.stdout:3906 write 0x65df70b thru 0x65e9c3a (0xa530 bytes) 2024-09-16T11:44:06.767 INFO:teuthology.orchestra.run.smithi070.stdout:3907 write 0x4781ced thru 0x4783bf1 (0x1f05 bytes) 2024-09-16T11:44:06.776 INFO:teuthology.orchestra.run.smithi070.stdout:3908 trunc from 0xdb679f0 to 0x8dffe8d 2024-09-16T11:44:06.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:44:06.787 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.787 INFO:teuthology.orchestra.run.smithi070.stdout:3910 punch from 0x639caa7 to 0x63a52ac, (0x8805 bytes) 2024-09-16T11:44:06.788 INFO:teuthology.orchestra.run.smithi070.stdout:3911 trunc from 0x8dffe8d to 0x9573dac 2024-09-16T11:44:06.801 INFO:teuthology.orchestra.run.smithi070.stdout:3912 write 0xb2e5def thru 0xb2f34e3 (0xd6f5 bytes) 2024-09-16T11:44:06.811 INFO:teuthology.orchestra.run.smithi070.stdout:3913 read 0x5950a5d thru 0x595529d (0x4841 bytes) 2024-09-16T11:44:06.814 INFO:teuthology.orchestra.run.smithi070.stdout:3914 trunc from 0xb2f34e4 to 0x8c0c379 2024-09-16T11:44:06.824 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:06.824 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:06.824 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:06.824 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:06.824 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:06.827 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:06.824+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:06.827 INFO:teuthology.orchestra.run.smithi070.stdout:3915 read 0xb7f9f8 thru 0xb87ede (0x84e7 bytes) 2024-09-16T11:44:06.827 INFO:teuthology.orchestra.run.smithi070.stdout:3916 writesame 0x7150437 thru 0x7155fc7 data_size 0x1(0x5b91 bytes) 2024-09-16T11:44:06.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:06.836 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.836 INFO:teuthology.orchestra.run.smithi070.stdout:3918 writesame 0x106f519 thru 0x1076227 data_size 0x1(0x6d0f bytes) 2024-09-16T11:44:06.857 INFO:teuthology.orchestra.run.smithi070.stdout:3919 write 0xc4c3172 thru 0xc4cf8c7 (0xc756 bytes) 2024-09-16T11:44:06.865 INFO:teuthology.orchestra.run.smithi070.stdout:3921 write 0x2cf09c2 thru 0x2cf83ea (0x7a29 bytes) 2024-09-16T11:44:06.872 INFO:teuthology.orchestra.run.smithi070.stdout:3922 read 0x5fc7ec6 thru 0x5fccd97 (0x4ed2 bytes) 2024-09-16T11:44:06.872 INFO:teuthology.orchestra.run.smithi070.stdout:3924 read 0xb265c47 thru 0xb26c145 (0x64ff bytes) 2024-09-16T11:44:06.872 INFO:teuthology.orchestra.run.smithi070.stdout:3925 write 0x760c3b0 thru 0x761b0e9 (0xed3a bytes) 2024-09-16T11:44:06.884 INFO:teuthology.orchestra.run.smithi070.stdout:3927 write 0xd4d2ea3 thru 0xd4e1ae3 (0xec41 bytes) 2024-09-16T11:44:06.893 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:06.894 INFO:teuthology.orchestra.run.smithi070.stdout:3930 read 0x95d411 thru 0x965037 (0x7c27 bytes) 2024-09-16T11:44:06.894 INFO:teuthology.orchestra.run.smithi070.stdout:3931 writesame 0x1f1c326 thru 0x1f23892 data_size 0x1(0x756d bytes) 2024-09-16T11:44:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T11:44:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:set 4.8 pg_upmap_items mapping to [3->6,5->1] 2024-09-16T11:44:06.914 INFO:teuthology.orchestra.run.smithi070.stdout:3934 read 0x1cec6a4 thru 0x1cfa676 (0xdfd3 bytes) 2024-09-16T11:44:06.914 INFO:teuthology.orchestra.run.smithi070.stdout:3935 write 0x3a5d5f1 thru 0x3a6c472 (0xee82 bytes) 2024-09-16T11:44:06.928 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:06.925+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:06.930 INFO:teuthology.orchestra.run.smithi070.stdout:3937 read 0x5ff57eb thru 0x5ff6940 (0x1156 bytes) 2024-09-16T11:44:06.930 INFO:teuthology.orchestra.run.smithi070.stdout:3938 write 0x60929fc thru 0x60956eb (0x2cf0 bytes) 2024-09-16T11:44:06.932 INFO:teuthology.orchestra.run.smithi070.stdout:3939 punch from 0x2874fc5 to 0x287b3de, (0x6419 bytes) 2024-09-16T11:44:06.932 INFO:teuthology.orchestra.run.smithi070.stdout:3940 read 0x2d076c4 thru 0x2d13269 (0xbba6 bytes) 2024-09-16T11:44:06.932 INFO:teuthology.orchestra.run.smithi070.stdout:3941 writesame 0x6a40adc thru 0x6a4fe62 data_size 0x9(0xf387 bytes) 2024-09-16T11:44:06.947 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:06.947 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:06.963 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:06.963 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:06.966 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:06.966 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:06.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:06.978 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:06.978 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:06.994 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:06.994 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:07.010 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:07.010 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:07.026 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:07.026 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:07.028 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:07.026+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:07.041 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:07.057 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:07.057 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:07.078 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:07.078 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:07.087 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:07.129 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:07.126+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:07.225 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:07.225 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:07.232 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:07.227+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:07.234 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:07.330 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:07.328+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:07.372 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:07.372 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:07.381 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:07.432 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:07.429+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:07.519 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:07.519 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:07.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:07.572 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:07.570+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:07.625 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:07.625 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:07.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:07.674 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:07.672+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:07.737 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:07.735+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:07.770 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:07.770 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:07.778 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:07.838 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:07.835+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:07.885 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:07.885 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:07.894 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:07.938 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:07.935+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:08.033 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:08.033 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:08.038 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:08.036+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:08.041 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:08.139 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:08.137+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:08.178 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:08.179 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:08.187 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:08.240 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:08.238+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:08.285 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:08.286 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:08.294 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:08.378 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:08.376+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:08.433 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:08.433 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:08.441 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:08.443 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:08.441+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:08.544 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:08.541+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:08.583 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:08.583 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:08.591 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:08.645 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:08.643+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:08.691 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:08.691 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:08.700 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:08.746 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:08.743+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:08.798 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:08.798 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:08.806 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:08.847 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:08.845+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:08.928 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:44:08.947 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:08.947 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:08.955 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:08.973 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:08.971+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:09.048 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:09.046+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:09.053 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:09.054 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:09.062 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:09.068 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:44:09.149 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:09.146+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:09.160 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:09.160 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:09.169 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:09.184 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:09.184 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:09.199 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:09.213 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:09.214 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:09.228 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:09.229 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:09.244 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:09.245 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:09.250 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:09.248+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:09.261 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:09.261 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:09.277 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:09.277 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:09.291 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:09.291 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:09.312 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:09.312 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:09.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:09.313 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:09.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.313 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:09.313 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:09.313 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.389061236000003, 2024-09-16T11:44:09.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:44:09.313 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.313 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.313 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:09.314 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:09.314 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:09.314 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:44:09.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.314 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:09.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:09.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:09.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:44:09.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:44:09.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:44:09.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:44:09.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:44:09.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:09.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:44:09.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:44:09.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:44:09.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:44:09.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:09.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:44:09.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:44:09.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.389015544000003, 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:09.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:44:09.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:44:09.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:44:09.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:44:09.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:44:09.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:44:09.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:44:09.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:44:09.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:44:09.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:44:09.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:09.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:44:09.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:44:09.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:44:09.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:44:09.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:09.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:44:09.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:44:09.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:44:09.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:44:09.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:44:09.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:44:09.326 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.326 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.327 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.327 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:09.327 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:09.327 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.388958614000003, 2024-09-16T11:44:09.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:44:09.328 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.328 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.328 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:09.328 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:09.328 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:09.328 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:44:09.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.329 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:09.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:09.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:44:09.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:09.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:44:09.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:44:09.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:44:09.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:44:09.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:44:09.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:44:09.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:44:09.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:44:09.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:44:09.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:44:09.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:09.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:44:09.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:44:09.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:44:09.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:44:09.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:09.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:44:09.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:44:09.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:44:09.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:44:09.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.388933246000001, 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:09.336 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:09.336 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:09.336 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:44:09.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.336 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:09.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:44:09.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:09.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:44:09.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:44:09.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:44:09.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:44:09.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:44:09.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:44:09.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.341 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.341 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:09.341 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:09.341 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.388881277999999, 2024-09-16T11:44:09.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:44:09.341 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:44:09.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:44:09.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:44:09.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:44:09.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:44:09.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:44:09.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:44:09.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:44:09.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:09.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:44:09.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:44:09.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:09.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:44:09.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:44:09.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:09.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:09.348 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.388800168000003, 2024-09-16T11:44:09.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:44:09.348 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.348 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:09.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:09.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:09.349 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:44:09.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.349 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:09.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:09.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:09.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:44:09.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:44:09.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:44:09.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:44:09.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:44:09.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:44:09.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:44:09.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:44:09.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:44:09.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:44:09.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:44:09.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:09.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:44:09.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:44:09.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:44:09.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:44:09.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:09.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:44:09.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:44:09.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:09.354 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.388774189999999, 2024-09-16T11:44:09.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:44:09.355 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.355 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:09.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:09.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:09.355 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:44:09.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.356 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:09.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:09.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:09.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:44:09.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:44:09.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:44:09.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:44:09.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:09.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:44:09.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:09.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:44:09.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:44:09.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:44:09.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:44:09.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:09.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:44:09.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:44:09.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:44:09.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:44:09.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:09.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:44:09.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:44:09.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:44:09.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:44:09.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:44:09.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:44:09.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.364 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.365 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:09.365 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:09.365 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.971255565, 2024-09-16T11:44:09.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:44:09.365 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.365 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:09.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:09.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:09.366 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:44:09.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.366 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:09.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:44:09.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:44:09.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:44:09.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:09.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:44:09.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:44:09.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:44:09.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.971233634000001, 2024-09-16T11:44:09.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:09.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:44:09.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:44:09.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:44:09.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:44:09.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:44:09.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:09.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:44:09.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:44:09.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:44:09.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:44:09.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:44:09.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:44:09.378 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.378 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.378 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.378 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:09.379 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:09.379 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.971156745999998, 2024-09-16T11:44:09.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:44:09.379 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.380 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:09.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:09.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:09.380 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:44:09.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.380 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:09.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:09.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:44:09.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:44:09.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:44:09.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:44:09.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:44:09.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:44:09.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:44:09.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:44:09.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:44:09.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:44:09.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:44:09.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:44:09.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:09.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:44:09.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:44:09.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:44:09.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:44:09.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:09.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:44:09.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:44:09.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:44:09.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:44:09.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:44:09.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:44:09.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.388 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.388 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28187 4.17 e52/28 4:ee916bd9:::rbd_data.11c1ce740b46.0000000000000000:head v 52'195, mlcod=52'193)", 2024-09-16T11:44:09.388 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:09.388 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.79973736, 2024-09-16T11:44:09.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075854429999999999, 2024-09-16T11:44:09.389 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.389 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.389 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:09.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:09.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509349+0000", 2024-09-16T11:44:09.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2300000000000001e-06 2024-09-16T11:44:09.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513315+0000", 2024-09-16T11:44:09.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039660590000000001 2024-09-16T11:44:09.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513328+0000", 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.308e-05 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513334+0000", 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513385+0000", 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1347000000000002e-05 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513407+0000", 2024-09-16T11:44:09.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1852999999999999e-05 2024-09-16T11:44:09.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516882+0000", 2024-09-16T11:44:09.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034749690000000001 2024-09-16T11:44:09.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516933+0000", 2024-09-16T11:44:09.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0952999999999998e-05 2024-09-16T11:44:09.392 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28959 4.8 e53/28 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head v 53'408, mlcod=53'406)", 2024-09-16T11:44:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.277588304, 2024-09-16T11:44:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088740420000000004, 2024-09-16T11:44:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:09.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:09.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031498+0000", 2024-09-16T11:44:09.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.652e-06 2024-09-16T11:44:09.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033515+0000", 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020165930000000001 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033539+0000", 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4295999999999999e-05 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033548+0000", 2024-09-16T11:44:09.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1239999999999994e-06 2024-09-16T11:44:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033610+0000", 2024-09-16T11:44:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1371999999999994e-05 2024-09-16T11:44:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033637+0000", 2024-09-16T11:44:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7098999999999999e-05 2024-09-16T11:44:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040341+0000", 2024-09-16T11:44:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067045819999999997 2024-09-16T11:44:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040371+0000", 2024-09-16T11:44:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9323999999999999e-05 2024-09-16T11:44:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32303 4.8 e53/28 4:1436ac82:::rbd_data.11c1cab081b3.0000000000000022:head v 53'417, mlcod=53'415)", 2024-09-16T11:44:09.398 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:09.398 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.8071623299999997, 2024-09-16T11:44:09.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085565160000000001, 2024-09-16T11:44:09.398 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.398 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.398 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:09.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:09.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:09.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-09-16T11:44:09.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501930+0000", 2024-09-16T11:44:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7649999999999996e-06 2024-09-16T11:44:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501946+0000", 2024-09-16T11:44:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5301000000000001e-05 2024-09-16T11:44:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501952+0000", 2024-09-16T11:44:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:44:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502100+0000", 2024-09-16T11:44:09.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014847600000000001 2024-09-16T11:44:09.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502120+0000", 2024-09-16T11:44:09.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9303000000000001e-05 2024-09-16T11:44:09.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510436+0000", 2024-09-16T11:44:09.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083166420000000008 2024-09-16T11:44:09.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510479+0000", 2024-09-16T11:44:09.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2979000000000003e-05 2024-09-16T11:44:09.402 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.402 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.402 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.403 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32306 4.1e e53/28 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head v 53'124, mlcod=53'122)", 2024-09-16T11:44:09.403 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:09.403 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.8070963359999999, 2024-09-16T11:44:09.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084990900000000008, 2024-09-16T11:44:09.403 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.403 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.403 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:09.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:09.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:09.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:44:09.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501994+0000", 2024-09-16T11:44:09.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:44:09.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502002+0000", 2024-09-16T11:44:09.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3839999999999997e-06 2024-09-16T11:44:09.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502007+0000", 2024-09-16T11:44:09.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6290000000000001e-06 2024-09-16T11:44:09.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502454+0000", 2024-09-16T11:44:09.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00044736800000000002 2024-09-16T11:44:09.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502471+0000", 2024-09-16T11:44:09.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6648999999999999e-05 2024-09-16T11:44:09.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510455+0000", 2024-09-16T11:44:09.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0079841909999999999 2024-09-16T11:44:09.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510488+0000", 2024-09-16T11:44:09.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2948999999999999e-05 2024-09-16T11:44:09.407 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.407 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.407 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32307 4.8 e53/28 4:11c1c37d:::rbd_data.11c1cab081b3.0000000000000026:head v 53'418, mlcod=53'415)", 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.8063410009999998, 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00776212, 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502750+0000", 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502760+0000", 2024-09-16T11:44:09.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:44:09.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502767+0000", 2024-09-16T11:44:09.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4219999999999997e-06 2024-09-16T11:44:09.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502802+0000", 2024-09-16T11:44:09.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5089999999999998e-05 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502816+0000", 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4092e-05 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510483+0000", 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00766704 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510506+0000", 2024-09-16T11:44:09.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3195999999999999e-05 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33583 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'416, mlcod=53'411)", 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.0983825920000001, 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078132459999999994, 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:09.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:09.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210703+0000", 2024-09-16T11:44:09.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:09.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210706+0000", 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-06 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210712+0000", 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210715+0000", 2024-09-16T11:44:09.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:44:09.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210736+0000", 2024-09-16T11:44:09.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0293999999999999e-05 2024-09-16T11:44:09.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210746+0000", 2024-09-16T11:44:09.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0178e-05 2024-09-16T11:44:09.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218477+0000", 2024-09-16T11:44:09.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007731501 2024-09-16T11:44:09.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218516+0000", 2024-09-16T11:44:09.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8148000000000001e-05 2024-09-16T11:44:09.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.416 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33584 4.0 e53/28 4:044e4d16:::rbd_data.11c1cab081b3.0000000000000019:head v 53'176, mlcod=53'174)", 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.0975711090000004, 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070059470000000002, 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.37e-07 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211524+0000", 2024-09-16T11:44:09.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4830000000000005e-06 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211535+0000", 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0842000000000001e-05 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211539+0000", 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9740000000000004e-06 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211584+0000", 2024-09-16T11:44:09.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5655000000000002e-05 2024-09-16T11:44:09.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211602+0000", 2024-09-16T11:44:09.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7756000000000001e-05 2024-09-16T11:44:09.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218497+0000", 2024-09-16T11:44:09.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0068944590000000004 2024-09-16T11:44:09.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218520+0000", 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3141000000000001e-05 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33590 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'419, mlcod=53'415)", 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.0873244509999997, 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082499879999999998, 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.421 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221761+0000", 2024-09-16T11:44:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-16T11:44:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221765+0000", 2024-09-16T11:44:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1590000000000003e-06 2024-09-16T11:44:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221774+0000", 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7490000000000004e-06 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221778+0000", 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0419999999999996e-06 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221800+0000", 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1926999999999999e-05 2024-09-16T11:44:09.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221810+0000", 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0079e-05 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.229980+0000", 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081698469999999992 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230010+0000", 2024-09-16T11:44:09.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0746e-05 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33591 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'420, mlcod=53'415)", 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.0867629670000003, 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007702576, 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222326+0000", 2024-09-16T11:44:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:44:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222337+0000", 2024-09-16T11:44:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1008e-05 2024-09-16T11:44:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222342+0000", 2024-09-16T11:44:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0050000000000004e-06 2024-09-16T11:44:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222366+0000", 2024-09-16T11:44:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3696000000000001e-05 2024-09-16T11:44:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222385+0000", 2024-09-16T11:44:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-16T11:44:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230012+0000", 2024-09-16T11:44:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076270940000000001 2024-09-16T11:44:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230025+0000", 2024-09-16T11:44:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2112e-05 2024-09-16T11:44:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33593 4.1e e53/28 4:7e77cec2:::rbd_data.11c1cab081b3.000000000000000c:head v 53'133, mlcod=53'131)", 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.0821290450000003, 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070516270000000004, 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226957+0000", 2024-09-16T11:44:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:44:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227014+0000", 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7074999999999998e-05 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227024+0000", 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0617e-05 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227030+0000", 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3279999999999996e-06 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227061+0000", 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1718999999999999e-05 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227074+0000", 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2738000000000001e-05 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.233975+0000", 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0069008480000000002 2024-09-16T11:44:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.234008+0000", 2024-09-16T11:44:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2614000000000003e-05 2024-09-16T11:44:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:09.435 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:09.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:09.436 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:09.381+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:09.485 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:09.483+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:09.534 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:09.535 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:09.535 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:09.535 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:09.543 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:09.586 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:09.584+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:09.622 INFO:teuthology.orchestra.run.smithi070.stdout:3942 writesame 0xd02e9b9 thru 0xd036b7f data_size 0x1(0x81c7 bytes) 2024-09-16T11:44:09.633 INFO:teuthology.orchestra.run.smithi070.stdout:3943 punch from 0x2299373 to 0x22a4ad3, (0xb760 bytes) 2024-09-16T11:44:09.633 INFO:teuthology.orchestra.run.smithi070.stdout:3945 write 0x218c8f2 thru 0x2198cbf (0xc3ce bytes) 2024-09-16T11:44:09.639 INFO:teuthology.orchestra.run.smithi070.stdout:3946 punch from 0x89f46e7 to 0x89fb5f8, (0x6f11 bytes) 2024-09-16T11:44:09.639 INFO:teuthology.orchestra.run.smithi070.stdout:3947 write 0x52bdcfc thru 0x52cae17 (0xd11c bytes) 2024-09-16T11:44:09.650 INFO:teuthology.orchestra.run.smithi070.stdout:3948 trunc from 0xd4e1ae4 to 0xcb395eb 2024-09-16T11:44:09.656 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:09.656 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:09.656 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:09.656 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:09.656 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:09.662 INFO:teuthology.orchestra.run.smithi070.stdout:3952 write 0x8f258da thru 0x8f327a8 (0xcecf bytes) 2024-09-16T11:44:09.667 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:09.673 INFO:teuthology.orchestra.run.smithi070.stdout:3953 write 0xedc6f03 thru 0xedd56c0 (0xe7be bytes) 2024-09-16T11:44:09.684 INFO:teuthology.orchestra.run.smithi070.stdout:3956 read 0xd90588e thru 0xd909334 (0x3aa7 bytes) 2024-09-16T11:44:09.684 INFO:teuthology.orchestra.run.smithi070.stdout:3958 writesame 0x6a46b1b thru 0x6a47dc4 data_size 0x1(0x12aa bytes) 2024-09-16T11:44:09.689 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:09.687+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:09.704 INFO:teuthology.orchestra.run.smithi070.stdout:3959 trunc from 0xedd56c1 to 0x17ff9ca 2024-09-16T11:44:09.738 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:09.738 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:09.738 INFO:teuthology.orchestra.run.smithi070.stdout:3965 read 0x7abebf thru 0x7bb7b5 (0xf8f7 bytes) 2024-09-16T11:44:09.739 INFO:teuthology.orchestra.run.smithi070.stdout:3966 punch from 0x95a034 to 0x963e38, (0x9e04 bytes) 2024-09-16T11:44:09.739 INFO:teuthology.orchestra.run.smithi070.stdout:3968 writesame 0x160047d thru 0x1607718 data_size 0x5a(0x729c bytes) 2024-09-16T11:44:09.748 INFO:teuthology.orchestra.run.smithi070.stdout:3970 trunc from 0x17ff9ca to 0x77a9724 2024-09-16T11:44:09.757 INFO:teuthology.orchestra.run.smithi070.stdout:3971 trunc from 0x77a9724 to 0x3ee07ef 2024-09-16T11:44:09.762 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:09.762 INFO:teuthology.orchestra.run.smithi070.stdout:3973 write 0x2354b65 thru 0x235b532 (0x69ce bytes) 2024-09-16T11:44:09.782 INFO:teuthology.orchestra.run.smithi070.stdout:3976 write 0xb520950 thru 0xb5262b6 (0x5967 bytes) 2024-09-16T11:44:09.787 INFO:teuthology.orchestra.run.smithi070.stdout:3977 read 0x8c6b6ec thru 0x8c6bca1 (0x5b6 bytes) 2024-09-16T11:44:09.788 INFO:teuthology.orchestra.run.smithi070.stdout:3978 read 0x29896cd thru 0x2998835 (0xf169 bytes) 2024-09-16T11:44:09.788 INFO:teuthology.orchestra.run.smithi070.stdout:3979 punch from 0x7fb692e to 0x7fb8672, (0x1d44 bytes) 2024-09-16T11:44:09.788 INFO:teuthology.orchestra.run.smithi070.stdout:3981 read 0xd56a28 thru 0xd5fde2 (0x93bb bytes) 2024-09-16T11:44:09.788 INFO:teuthology.orchestra.run.smithi070.stdout:3982 read 0x993140d thru 0x99402cc (0xeec0 bytes) 2024-09-16T11:44:09.788 INFO:teuthology.orchestra.run.smithi070.stdout:3983 writesame 0x66c5d9b thru 0x66ccead data_size 0x1(0x7113 bytes) 2024-09-16T11:44:09.789 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:09.787+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:09.801 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:09.801 INFO:teuthology.orchestra.run.smithi070.stdout:3985 write 0x6194c91 thru 0x61a2ce5 (0xe055 bytes) 2024-09-16T11:44:09.808 INFO:teuthology.orchestra.run.smithi070.stdout:3987 writesame 0x5a1dabf thru 0x5a2adc8 data_size 0x77(0xd30a bytes) 2024-09-16T11:44:09.810 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:09.810 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:09.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:09.811 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:09.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.811 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:5357 4.b 4:d70b03a5:::rbd_data.11c18a2497a7.0000000000000026:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:44:09.811 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:41.982429+0000", 2024-09-16T11:44:09.811 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 27.825043055999998, 2024-09-16T11:44:09.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.018203652000000001, 2024-09-16T11:44:09.811 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.811 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.812 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5357 2024-09-16T11:44:09.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.812 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982429+0000", 2024-09-16T11:44:09.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982429+0000", 2024-09-16T11:44:09.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982431+0000", 2024-09-16T11:44:09.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.705e-06 2024-09-16T11:44:09.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982438+0000", 2024-09-16T11:44:09.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.588e-06 2024-09-16T11:44:09.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982454+0000", 2024-09-16T11:44:09.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5951e-05 2024-09-16T11:44:09.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982460+0000", 2024-09-16T11:44:09.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9329999999999998e-06 2024-09-16T11:44:09.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982515+0000", 2024-09-16T11:44:09.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4583000000000002e-05 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.982609+0000", 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3857999999999995e-05 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:41.985720+0000", 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031111020000000001 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000561+0000", 2024-09-16T11:44:09.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014841221999999999 2024-09-16T11:44:09.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:09.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000571+0000", 2024-09-16T11:44:09.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8039999999999996e-06 2024-09-16T11:44:09.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:09.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000606+0000", 2024-09-16T11:44:09.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5197999999999997e-05 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:42.000633+0000", 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6707999999999999e-05 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:9081 4.b 4:d317bc5b:::rbd_data.11c1dc795e15.000000000000000d:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 23.753516593000001, 2024-09-16T11:44:09.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027796406999999999, 2024-09-16T11:44:09.818 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.818 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.818 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.818 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.818 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.818 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 9081 2024-09-16T11:44:09.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.818 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:44:09.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:44:09.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053957+0000", 2024-09-16T11:44:09.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7010000000000001e-06 2024-09-16T11:44:09.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053965+0000", 2024-09-16T11:44:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5939999999999996e-06 2024-09-16T11:44:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053980+0000", 2024-09-16T11:44:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5099000000000001e-05 2024-09-16T11:44:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053986+0000", 2024-09-16T11:44:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5759999999999996e-06 2024-09-16T11:44:09.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.054023+0000", 2024-09-16T11:44:09.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999999e-05 2024-09-16T11:44:09.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.054076+0000", 2024-09-16T11:44:09.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2852999999999997e-05 2024-09-16T11:44:09.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.058880+0000", 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048042639999999999 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081663+0000", 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022782908000000001 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081672+0000", 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0440000000000001e-06 2024-09-16T11:44:09.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:09.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081707+0000", 2024-09-16T11:44:09.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4986e-05 2024-09-16T11:44:09.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081752+0000", 2024-09-16T11:44:09.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4781999999999999e-05 2024-09-16T11:44:09.823 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.823 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.823 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.824 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:9113 4.b 4:d317bc5b:::rbd_data.11c1dc795e15.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:44:09.824 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.119551+0000", 2024-09-16T11:44:09.824 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 23.687921134, 2024-09-16T11:44:09.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.020116319000000001, 2024-09-16T11:44:09.824 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.824 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.824 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.824 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.824 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.824 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 9113 2024-09-16T11:44:09.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.824 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119551+0000", 2024-09-16T11:44:09.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119551+0000", 2024-09-16T11:44:09.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119553+0000", 2024-09-16T11:44:09.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.782e-06 2024-09-16T11:44:09.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119563+0000", 2024-09-16T11:44:09.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0325e-05 2024-09-16T11:44:09.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119577+0000", 2024-09-16T11:44:09.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.382e-05 2024-09-16T11:44:09.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119583+0000", 2024-09-16T11:44:09.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.57e-06 2024-09-16T11:44:09.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119620+0000", 2024-09-16T11:44:09.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7737000000000001e-05 2024-09-16T11:44:09.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.119673+0000", 2024-09-16T11:44:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3152000000000002e-05 2024-09-16T11:44:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:44:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.122945+0000", 2024-09-16T11:44:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0032719390000000002 2024-09-16T11:44:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139568+0000", 2024-09-16T11:44:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016622839 2024-09-16T11:44:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139606+0000", 2024-09-16T11:44:09.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8006000000000002e-05 2024-09-16T11:44:09.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:09.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139641+0000", 2024-09-16T11:44:09.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5076000000000001e-05 2024-09-16T11:44:09.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.139667+0000", 2024-09-16T11:44:09.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6072999999999999e-05 2024-09-16T11:44:09.832 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.832 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.832 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.832 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32306 4.1e 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:09.832 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.501566+0000", 2024-09-16T11:44:09.832 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 8.3059059560000001, 2024-09-16T11:44:09.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093950460000000006, 2024-09-16T11:44:09.832 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.832 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.832 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.833 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.833 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.833 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32306 2024-09-16T11:44:09.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.833 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501566+0000", 2024-09-16T11:44:09.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501566+0000", 2024-09-16T11:44:09.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.834 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.834 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501569+0000", 2024-09-16T11:44:09.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3290000000000001e-06 2024-09-16T11:44:09.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.834 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.834 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501576+0000", 2024-09-16T11:44:09.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9829999999999999e-06 2024-09-16T11:44:09.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501595+0000", 2024-09-16T11:44:09.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9548e-05 2024-09-16T11:44:09.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501603+0000", 2024-09-16T11:44:09.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.729e-06 2024-09-16T11:44:09.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501664+0000", 2024-09-16T11:44:09.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1161000000000006e-05 2024-09-16T11:44:09.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501747+0000", 2024-09-16T11:44:09.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2888999999999999e-05 2024-09-16T11:44:09.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:09.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.501854+0000", 2024-09-16T11:44:09.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010701600000000001 2024-09-16T11:44:09.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.502516+0000", 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066228299999999995 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.510891+0000", 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0083744539999999999 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.510942+0000", 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1325000000000001e-05 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.510961+0000", 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9329000000000001e-05 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32411 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515676+0000", 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 8.2917958689999995, 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093416420000000007, 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.838 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32411 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515676+0000", 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.839 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515676+0000", 2024-09-16T11:44:09.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515677+0000", 2024-09-16T11:44:09.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:44:09.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515678+0000", 2024-09-16T11:44:09.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6729999999999999e-06 2024-09-16T11:44:09.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515684+0000", 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515687+0000", 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2650000000000001e-06 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524945+0000", 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0092578490000000003 2024-09-16T11:44:09.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.524982+0000", 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7392999999999999e-05 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525018+0000", 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5645000000000002e-05 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.842 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32412 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:09.843 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515691+0000", 2024-09-16T11:44:09.843 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 8.291781576, 2024-09-16T11:44:09.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0094970969999999995, 2024-09-16T11:44:09.843 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.843 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.843 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.843 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.843 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.843 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32412 2024-09-16T11:44:09.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.843 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515691+0000", 2024-09-16T11:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515691+0000", 2024-09-16T11:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515691+0000", 2024-09-16T11:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3099999999999998e-07 2024-09-16T11:44:09.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515693+0000", 2024-09-16T11:44:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6610000000000001e-06 2024-09-16T11:44:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515698+0000", 2024-09-16T11:44:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.592e-06 2024-09-16T11:44:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515702+0000", 2024-09-16T11:44:09.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.326e-06 2024-09-16T11:44:09.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525046+0000", 2024-09-16T11:44:09.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093440069999999997 2024-09-16T11:44:09.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525125+0000", 2024-09-16T11:44:09.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9629000000000006e-05 2024-09-16T11:44:09.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525188+0000", 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2351e-05 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32413 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515705+0000", 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 8.2917669220000008, 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0095604680000000008, 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.847 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32413 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515705+0000", 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515705+0000", 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515706+0000", 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2599999999999998e-07 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515707+0000", 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6440000000000001e-06 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515713+0000", 2024-09-16T11:44:09.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7679999999999997e-06 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515716+0000", 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.349e-06 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525198+0000", 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0094821130000000003 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525233+0000", 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4430000000000001e-05 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525266+0000", 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2838e-05 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32414 4.16 4:6fe67e2e:::rbd_data.11c1ce740b46.000000000000001b:16 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.515720+0000", 2024-09-16T11:44:09.851 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 8.2917521510000007, 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0097012409999999993, 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32414 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515720+0000", 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515720+0000", 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515720+0000", 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515722+0000", 2024-09-16T11:44:09.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8369999999999999e-06 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515728+0000", 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.524e-06 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.515731+0000", 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3060000000000002e-06 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525284+0000", 2024-09-16T11:44:09.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0095529299999999994 2024-09-16T11:44:09.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525359+0000", 2024-09-16T11:44:09.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5087999999999997e-05 2024-09-16T11:44:09.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.525421+0000", 2024-09-16T11:44:09.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2126999999999997e-05 2024-09-16T11:44:09.855 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.855 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.856 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.856 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:32718 4.3 4:c5c936b1:::rbd_data.11c1cab081b3.0000000000000009:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:09.856 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:01.595459+0000", 2024-09-16T11:44:09.856 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 8.2120127449999991, 2024-09-16T11:44:09.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0094432419999999993, 2024-09-16T11:44:09.856 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.856 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:09.856 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.856 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.856 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.856 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 32718 2024-09-16T11:44:09.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.857 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595459+0000", 2024-09-16T11:44:09.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595459+0000", 2024-09-16T11:44:09.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595460+0000", 2024-09-16T11:44:09.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.54e-07 2024-09-16T11:44:09.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595464+0000", 2024-09-16T11:44:09.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1710000000000004e-06 2024-09-16T11:44:09.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595477+0000", 2024-09-16T11:44:09.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2197e-05 2024-09-16T11:44:09.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595482+0000", 2024-09-16T11:44:09.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0130000000000001e-06 2024-09-16T11:44:09.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595526+0000", 2024-09-16T11:44:09.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4363000000000002e-05 2024-09-16T11:44:09.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.595609+0000", 2024-09-16T11:44:09.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2641999999999999e-05 2024-09-16T11:44:09.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5", 2024-09-16T11:44:09.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.597861+0000", 2024-09-16T11:44:09.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022526059999999999 2024-09-16T11:44:09.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:09.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.604625+0000", 2024-09-16T11:44:09.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.006764117 2024-09-16T11:44:09.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:09.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.604839+0000", 2024-09-16T11:44:09.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000213663 2024-09-16T11:44:09.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:09.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.604869+0000", 2024-09-16T11:44:09.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9688000000000001e-05 2024-09-16T11:44:09.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:01.604903+0000", 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3927999999999998e-05 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:41483 4.6 4:66e9c274:::rbd_data.11c1bd2a98f3.0000000000000010:1a [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:06.393735+0000", 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.4137375159999999, 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093158100000000008, 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.862 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 41483 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393735+0000", 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393735+0000", 2024-09-16T11:44:09.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393735+0000", 2024-09-16T11:44:09.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:09.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393737+0000", 2024-09-16T11:44:09.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7209999999999999e-06 2024-09-16T11:44:09.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393743+0000", 2024-09-16T11:44:09.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:44:09.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393746+0000", 2024-09-16T11:44:09.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.428e-06 2024-09-16T11:44:09.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.402888+0000", 2024-09-16T11:44:09.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0091420129999999992 2024-09-16T11:44:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.402976+0000", 2024-09-16T11:44:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8138000000000002e-05 2024-09-16T11:44:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403050+0000", 2024-09-16T11:44:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4399000000000004e-05 2024-09-16T11:44:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:41484 4.6 4:66e9c274:::rbd_data.11c1bd2a98f3.0000000000000010:1a [read 2859008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:06.393750+0000", 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.4137226630000002, 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0094747310000000001, 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 41484 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.867 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393750+0000", 2024-09-16T11:44:09.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393750+0000", 2024-09-16T11:44:09.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393750+0000", 2024-09-16T11:44:09.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-07 2024-09-16T11:44:09.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393751+0000", 2024-09-16T11:44:09.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5239999999999999e-06 2024-09-16T11:44:09.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393756+0000", 2024-09-16T11:44:09.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0549999999999999e-06 2024-09-16T11:44:09.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393761+0000", 2024-09-16T11:44:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1670000000000001e-06 2024-09-16T11:44:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403077+0000", 2024-09-16T11:44:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093161649999999995 2024-09-16T11:44:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403156+0000", 2024-09-16T11:44:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9081999999999999e-05 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403224+0000", 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8428000000000003e-05 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:41485 4.6 4:66e9c274:::rbd_data.11c1bd2a98f3.0000000000000010:1a [read 2891776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:06.393764+0000", 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.4137077539999998, 2024-09-16T11:44:09.871 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0096447140000000004, 2024-09-16T11:44:09.872 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.872 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.872 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.872 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.872 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.872 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 41485 2024-09-16T11:44:09.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.872 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.872 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.872 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393764+0000", 2024-09-16T11:44:09.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393764+0000", 2024-09-16T11:44:09.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393765+0000", 2024-09-16T11:44:09.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3399999999999999e-07 2024-09-16T11:44:09.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393767+0000", 2024-09-16T11:44:09.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.75e-06 2024-09-16T11:44:09.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393774+0000", 2024-09-16T11:44:09.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2339999999999997e-06 2024-09-16T11:44:09.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393778+0000", 2024-09-16T11:44:09.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:44:09.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403250+0000", 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.009472411 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403336+0000", 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6149000000000005e-05 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403409+0000", 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2743000000000005e-05 2024-09-16T11:44:09.875 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.876 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.876 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.876 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:41486 4.6 4:66e9c274:::rbd_data.11c1bd2a98f3.0000000000000010:1a [read 3055616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:09.876 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:06.393782+0000", 2024-09-16T11:44:09.876 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.4136905290000001, 2024-09-16T11:44:09.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0097943849999999992, 2024-09-16T11:44:09.876 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.876 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.876 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.877 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.877 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.877 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 41486 2024-09-16T11:44:09.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.877 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.877 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393782+0000", 2024-09-16T11:44:09.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.877 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393782+0000", 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393782+0000", 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393784+0000", 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.858e-06 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393790+0000", 2024-09-16T11:44:09.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6269999999999997e-06 2024-09-16T11:44:09.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393793+0000", 2024-09-16T11:44:09.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.534e-06 2024-09-16T11:44:09.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403440+0000", 2024-09-16T11:44:09.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0096470099999999993 2024-09-16T11:44:09.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.880 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403512+0000", 2024-09-16T11:44:09.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2177999999999999e-05 2024-09-16T11:44:09.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.880 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403576+0000", 2024-09-16T11:44:09.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3804000000000001e-05 2024-09-16T11:44:09.880 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.880 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.880 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:41487 4.6 4:66e9c274:::rbd_data.11c1bd2a98f3.0000000000000010:1a [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:06.393797+0000", 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.4136755640000001, 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0099479630000000006, 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 41487 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393797+0000", 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393797+0000", 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393797+0000", 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:44:09.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393799+0000", 2024-09-16T11:44:09.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:44:09.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393808+0000", 2024-09-16T11:44:09.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9900000000000003e-06 2024-09-16T11:44:09.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393811+0000", 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7019999999999999e-06 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403605+0000", 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0097938250000000008 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403683+0000", 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7311000000000006e-05 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403745+0000", 2024-09-16T11:44:09.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2021999999999997e-05 2024-09-16T11:44:09.885 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.885 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.885 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.885 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:41488 4.6 4:66e9c274:::rbd_data.11c1bd2a98f3.0000000000000010:1a [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:09.885 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:06.393818+0000", 2024-09-16T11:44:09.885 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.4136539699999999, 2024-09-16T11:44:09.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010087741000000001, 2024-09-16T11:44:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 41488 2024-09-16T11:44:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393818+0000", 2024-09-16T11:44:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393818+0000", 2024-09-16T11:44:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393819+0000", 2024-09-16T11:44:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:44:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393821+0000", 2024-09-16T11:44:09.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.007e-06 2024-09-16T11:44:09.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393834+0000", 2024-09-16T11:44:09.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3193000000000001e-05 2024-09-16T11:44:09.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393838+0000", 2024-09-16T11:44:09.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9840000000000003e-06 2024-09-16T11:44:09.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.889 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403771+0000", 2024-09-16T11:44:09.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0099332529999999995 2024-09-16T11:44:09.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.889 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403839+0000", 2024-09-16T11:44:09.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7627999999999997e-05 2024-09-16T11:44:09.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.890 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403906+0000", 2024-09-16T11:44:09.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7251999999999999e-05 2024-09-16T11:44:09.890 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.890 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.890 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.890 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:41489 4.6 4:66e9c274:::rbd_data.11c1bd2a98f3.0000000000000010:1a [read 3284992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:09.890 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:06.393843+0000", 2024-09-16T11:44:09.890 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.4136291399999998, 2024-09-16T11:44:09.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.01022437, 2024-09-16T11:44:09.890 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.891 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.891 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.891 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.891 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.891 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 41489 2024-09-16T11:44:09.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.891 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393843+0000", 2024-09-16T11:44:09.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393843+0000", 2024-09-16T11:44:09.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393843+0000", 2024-09-16T11:44:09.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2e-07 2024-09-16T11:44:09.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393846+0000", 2024-09-16T11:44:09.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0279999999999999e-06 2024-09-16T11:44:09.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393852+0000", 2024-09-16T11:44:09.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8519999999999998e-06 2024-09-16T11:44:09.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393856+0000", 2024-09-16T11:44:09.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7239999999999998e-06 2024-09-16T11:44:09.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.403937+0000", 2024-09-16T11:44:09.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010081025 2024-09-16T11:44:09.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404006+0000", 2024-09-16T11:44:09.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9173999999999999e-05 2024-09-16T11:44:09.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.895 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.895 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404067+0000", 2024-09-16T11:44:09.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1147000000000003e-05 2024-09-16T11:44:09.895 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.895 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.895 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.895 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:41490 4.6 4:66e9c274:::rbd_data.11c1bd2a98f3.0000000000000010:1a [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:09.895 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:06.393860+0000", 2024-09-16T11:44:09.895 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.4136124130000001, 2024-09-16T11:44:09.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010381239, 2024-09-16T11:44:09.895 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.896 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.896 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 41490 2024-09-16T11:44:09.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.896 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.896 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.896 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393860+0000", 2024-09-16T11:44:09.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393860+0000", 2024-09-16T11:44:09.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393860+0000", 2024-09-16T11:44:09.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:44:09.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393862+0000", 2024-09-16T11:44:09.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.702e-06 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393869+0000", 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7329999999999997e-06 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393872+0000", 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.557e-06 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404099+0000", 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010226565 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404169+0000", 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0542000000000003e-05 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404241+0000", 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1759000000000002e-05 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.899 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.900 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:41491 4.6 4:66e9c274:::rbd_data.11c1bd2a98f3.0000000000000010:1a [read 3481600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:09.900 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:06.393876+0000", 2024-09-16T11:44:09.900 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.4135965819999998, 2024-09-16T11:44:09.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010527729, 2024-09-16T11:44:09.900 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.900 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.900 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.900 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.901 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.901 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 41491 2024-09-16T11:44:09.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.901 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393876+0000", 2024-09-16T11:44:09.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393876+0000", 2024-09-16T11:44:09.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393876+0000", 2024-09-16T11:44:09.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3099999999999999e-07 2024-09-16T11:44:09.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393878+0000", 2024-09-16T11:44:09.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8020000000000001e-06 2024-09-16T11:44:09.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393884+0000", 2024-09-16T11:44:09.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7629999999999997e-06 2024-09-16T11:44:09.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393887+0000", 2024-09-16T11:44:09.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.41e-06 2024-09-16T11:44:09.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404268+0000", 2024-09-16T11:44:09.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010381244 2024-09-16T11:44:09.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404333+0000", 2024-09-16T11:44:09.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4534000000000002e-05 2024-09-16T11:44:09.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404403+0000", 2024-09-16T11:44:09.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0644999999999999e-05 2024-09-16T11:44:09.904 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.904 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.905 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.905 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:41492 4.6 4:66e9c274:::rbd_data.11c1bd2a98f3.0000000000000010:1a [read 3645440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:09.905 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:06.393890+0000", 2024-09-16T11:44:09.905 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.413581813, 2024-09-16T11:44:09.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010844789000000001, 2024-09-16T11:44:09.905 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.905 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.905 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.905 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.905 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 41492 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393890+0000", 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393890+0000", 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393891+0000", 2024-09-16T11:44:09.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.53e-07 2024-09-16T11:44:09.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393893+0000", 2024-09-16T11:44:09.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:44:09.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393898+0000", 2024-09-16T11:44:09.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2220000000000001e-06 2024-09-16T11:44:09.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393901+0000", 2024-09-16T11:44:09.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4699999999999998e-06 2024-09-16T11:44:09.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404428+0000", 2024-09-16T11:44:09.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010527175 2024-09-16T11:44:09.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404491+0000", 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2216000000000002e-05 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404735+0000", 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024458500000000002 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:41493 4.6 4:66e9c274:::rbd_data.11c1bd2a98f3.0000000000000010:1a [read 3678208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:06.393905+0000", 2024-09-16T11:44:09.909 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.4135667600000001, 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010789973, 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 41493 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393905+0000", 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:09.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393905+0000", 2024-09-16T11:44:09.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:09.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:09.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393906+0000", 2024-09-16T11:44:09.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:44:09.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:09.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393908+0000", 2024-09-16T11:44:09.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.821e-06 2024-09-16T11:44:09.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:09.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393915+0000", 2024-09-16T11:44:09.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.357e-06 2024-09-16T11:44:09.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:09.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.393919+0000", 2024-09-16T11:44:09.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.631e-06 2024-09-16T11:44:09.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:09.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404573+0000", 2024-09-16T11:44:09.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010654100999999999 2024-09-16T11:44:09.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:09.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404637+0000", 2024-09-16T11:44:09.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4012999999999999e-05 2024-09-16T11:44:09.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:09.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:09.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:09.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:06.404695+0000", 2024-09-16T11:44:09.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8678999999999998e-05 2024-09-16T11:44:09.914 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.914 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.914 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.914 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:09.914 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:09.914 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:09.915 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:09.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:09.918 INFO:teuthology.orchestra.run.smithi070.stdout:3990 writesame 0xbfa0415 thru 0xbfaef4a data_size 0x2e(0xeb36 bytes) 2024-09-16T11:44:09.918 INFO:teuthology.orchestra.run.smithi070.stdout:3991 punch from 0x7872f2c to 0x787f204, (0xc2d8 bytes) 2024-09-16T11:44:09.918 INFO:teuthology.orchestra.run.smithi070.stdout:3992 trunc from 0xbfaef4b to 0x58ac893 2024-09-16T11:44:09.918 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:09.918 INFO:teuthology.orchestra.run.smithi070.stdout:3994 trunc from 0x58ac893 to 0xaf5a19b 2024-09-16T11:44:09.918 INFO:teuthology.orchestra.run.smithi070.stdout:3995 write 0x4ae77 thru 0x53e37 (0x8fc1 bytes) 2024-09-16T11:44:09.918 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:09.918 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:09.918 INFO:teuthology.orchestra.run.smithi070.stdout:3998 write 0x72c5169 thru 0x72ceaa4 (0x993c bytes) 2024-09-16T11:44:09.919 INFO:teuthology.orchestra.run.smithi070.stdout:4000 writesame 0x7756f05 thru 0x7757c7e data_size 0xf(0xd7a bytes) 2024-09-16T11:44:09.919 INFO:teuthology.orchestra.run.smithi070.stdout:4001 write 0x8b9abb7 thru 0x8b9c2ac (0x16f6 bytes) 2024-09-16T11:44:09.919 INFO:teuthology.orchestra.run.smithi070.stdout:4002 punch from 0x66a17aa to 0x66a86b4, (0x6f0a bytes) 2024-09-16T11:44:09.919 INFO:teuthology.orchestra.run.smithi070.stdout:4003 read 0x75a28c thru 0x75dee8 (0x3c5d bytes) 2024-09-16T11:44:09.919 INFO:teuthology.orchestra.run.smithi070.stdout:4004 read 0x14a708c thru 0x14af4de (0x8453 bytes) 2024-09-16T11:44:09.919 INFO:teuthology.orchestra.run.smithi070.stdout:4005 writesame 0x753475a thru 0x7536b7f data_size 0x7(0x2426 bytes) 2024-09-16T11:44:09.919 INFO:teuthology.orchestra.run.smithi070.stdout:4006 read 0x5a4bcb4 thru 0x5a56fbf (0xb30c bytes) 2024-09-16T11:44:09.919 INFO:teuthology.orchestra.run.smithi070.stdout:4007 read 0x3658fc5 thru 0x365d7bf (0x47fb bytes) 2024-09-16T11:44:09.919 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:09.919 INFO:teuthology.orchestra.run.smithi070.stdout:4009 punch from 0xac6b3b to 0xad25aa, (0xba6f bytes) 2024-09-16T11:44:09.919 INFO:teuthology.orchestra.run.smithi070.stdout:4010 punch from 0x41339d9 to 0x4140e63, (0xd48a bytes) 2024-09-16T11:44:09.919 INFO:teuthology.orchestra.run.smithi070.stdout:4012 clone 24 order 23 su 65536 sc 8 2024-09-16T11:44:09.920 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T11:44:09.920 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:09.913+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:09.976 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone23 from 0xaf5a19b (overlap 0x954132) to 0x598d84 2024-09-16T11:44:10.016 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:10.014+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:10.071 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:10.071 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:10.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.071 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:44111 4.8 4.7e2fcfe8 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:44:10.071 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:10.071 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.086623773000000001, 2024-09-16T11:44:10.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.086644611999999996, 2024-09-16T11:44:10.072 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.072 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for peered", 2024-09-16T11:44:10.072 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.072 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.072 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.072 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 44111 2024-09-16T11:44:10.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.072 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:10.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:10.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:10.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7199999999999998e-07 2024-09-16T11:44:10.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983239+0000", 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9879999999999998e-06 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983248+0000", 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.371e-06 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983253+0000", 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1150000000000002e-06 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983303+0000", 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0380000000000002e-05 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983311+0000", 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7519999999999996e-06 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:10.075 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 1 2024-09-16T11:44:10.076 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:10.079 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:10.117 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:10.114+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:10.217 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:10.218 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:10.218 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:10.218 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:10.218 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:10.218 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:10.216+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:10.226 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:10.319 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:10.317+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:10.325 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:10.325 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:10.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:10.325 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:10.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.325 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19182 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3932160~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.325 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.507225+0000", 2024-09-16T11:44:10.325 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.815224446, 2024-09-16T11:44:10.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048909983999999997, 2024-09-16T11:44:10.326 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.326 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.326 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.326 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.326 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.326 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19182 2024-09-16T11:44:10.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.326 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.507225+0000", 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.507225+0000", 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.509102+0000", 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001877213 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.509107+0000", 2024-09-16T11:44:10.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1950000000000002e-06 2024-09-16T11:44:10.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.509117+0000", 2024-09-16T11:44:10.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8619999999999998e-06 2024-09-16T11:44:10.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.509122+0000", 2024-09-16T11:44:10.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:44:10.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.556023+0000", 2024-09-16T11:44:10.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.046900305000000003 2024-09-16T11:44:10.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.556064+0000", 2024-09-16T11:44:10.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1859e-05 2024-09-16T11:44:10.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.556135+0000", 2024-09-16T11:44:10.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0241999999999996e-05 2024-09-16T11:44:10.330 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.330 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.330 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.330 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19364 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1171456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.330 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:10.331 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.809171315, 2024-09-16T11:44:10.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049001805000000002, 2024-09-16T11:44:10.331 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.331 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.331 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.331 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.331 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.331 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19364 2024-09-16T11:44:10.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.331 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:10.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:10.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:10.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:44:10.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513280+0000", 2024-09-16T11:44:10.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7090000000000001e-06 2024-09-16T11:44:10.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513286+0000", 2024-09-16T11:44:10.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5840000000000002e-06 2024-09-16T11:44:10.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513290+0000", 2024-09-16T11:44:10.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3680000000000004e-06 2024-09-16T11:44:10.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562121+0000", 2024-09-16T11:44:10.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048831324000000002 2024-09-16T11:44:10.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562195+0000", 2024-09-16T11:44:10.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3415000000000001e-05 2024-09-16T11:44:10.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562280+0000", 2024-09-16T11:44:10.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4975999999999994e-05 2024-09-16T11:44:10.335 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.336 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.336 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.336 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19365 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1490944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.336 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:10.336 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.809151576, 2024-09-16T11:44:10.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049335349000000001, 2024-09-16T11:44:10.336 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.336 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.336 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.336 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.336 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.337 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19365 2024-09-16T11:44:10.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.337 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:10.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:10.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:10.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:10.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513300+0000", 2024-09-16T11:44:10.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3020000000000002e-06 2024-09-16T11:44:10.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513306+0000", 2024-09-16T11:44:10.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8780000000000003e-06 2024-09-16T11:44:10.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513310+0000", 2024-09-16T11:44:10.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1309999999999999e-06 2024-09-16T11:44:10.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562322+0000", 2024-09-16T11:44:10.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049011635999999997 2024-09-16T11:44:10.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562544+0000", 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022168000000000001 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562633+0000", 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9350000000000003e-05 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.341 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19366 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1916928~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.341 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:10.341 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.809134644, 2024-09-16T11:44:10.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049865073000000003, 2024-09-16T11:44:10.341 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.341 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.341 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.341 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.341 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.341 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19366 2024-09-16T11:44:10.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.341 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:10.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:10.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515102+0000", 2024-09-16T11:44:10.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017877419999999999 2024-09-16T11:44:10.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515106+0000", 2024-09-16T11:44:10.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4450000000000001e-06 2024-09-16T11:44:10.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515113+0000", 2024-09-16T11:44:10.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7680000000000008e-06 2024-09-16T11:44:10.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515117+0000", 2024-09-16T11:44:10.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:44:10.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562756+0000", 2024-09-16T11:44:10.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047638579 2024-09-16T11:44:10.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563093+0000", 2024-09-16T11:44:10.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000337516 2024-09-16T11:44:10.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563180+0000", 2024-09-16T11:44:10.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6420000000000003e-05 2024-09-16T11:44:10.345 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.345 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.345 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.346 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19371 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3194880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.346 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:10.346 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.807237437, 2024-09-16T11:44:10.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048948356999999998, 2024-09-16T11:44:10.346 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.346 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.346 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.346 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.346 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.346 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19371 2024-09-16T11:44:10.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.346 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:10.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:10.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:10.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:10.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515214+0000", 2024-09-16T11:44:10.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5969999999999999e-06 2024-09-16T11:44:10.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515219+0000", 2024-09-16T11:44:10.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4559999999999999e-06 2024-09-16T11:44:10.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515223+0000", 2024-09-16T11:44:10.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:44:10.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563991+0000", 2024-09-16T11:44:10.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048768911999999998 2024-09-16T11:44:10.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564063+0000", 2024-09-16T11:44:10.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2010999999999999e-05 2024-09-16T11:44:10.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564160+0000", 2024-09-16T11:44:10.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6617e-05 2024-09-16T11:44:10.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.350 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.351 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19372 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3620864~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.351 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:10.351 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.807223220000001, 2024-09-16T11:44:10.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049130502, 2024-09-16T11:44:10.351 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.351 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.351 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19372 2024-09-16T11:44:10.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.352 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:10.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:10.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:10.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:44:10.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515228+0000", 2024-09-16T11:44:10.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T11:44:10.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515233+0000", 2024-09-16T11:44:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:44:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515237+0000", 2024-09-16T11:44:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.27e-06 2024-09-16T11:44:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564199+0000", 2024-09-16T11:44:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048961787999999999 2024-09-16T11:44:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564275+0000", 2024-09-16T11:44:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6001000000000006e-05 2024-09-16T11:44:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564356+0000", 2024-09-16T11:44:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1941999999999996e-05 2024-09-16T11:44:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.356 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19373 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3940352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.356 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:10.356 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.80720882, 2024-09-16T11:44:10.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049289588000000002, 2024-09-16T11:44:10.356 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.356 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.356 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.357 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.357 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.357 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19373 2024-09-16T11:44:10.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.357 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:10.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:10.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515241+0000", 2024-09-16T11:44:10.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:44:10.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515242+0000", 2024-09-16T11:44:10.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.528e-06 2024-09-16T11:44:10.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515248+0000", 2024-09-16T11:44:10.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:10.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515251+0000", 2024-09-16T11:44:10.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2480000000000001e-06 2024-09-16T11:44:10.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564388+0000", 2024-09-16T11:44:10.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049137313000000002 2024-09-16T11:44:10.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564455+0000", 2024-09-16T11:44:10.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7049000000000001e-05 2024-09-16T11:44:10.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564530+0000", 2024-09-16T11:44:10.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4626e-05 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19374 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 4046848~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.807194417, 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049457133, 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.361 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19374 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515257+0000", 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5740000000000001e-06 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515262+0000", 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2399999999999998e-06 2024-09-16T11:44:10.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515265+0000", 2024-09-16T11:44:10.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3909999999999998e-06 2024-09-16T11:44:10.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564555+0000", 2024-09-16T11:44:10.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049290002999999999 2024-09-16T11:44:10.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564628+0000", 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2559999999999996e-05 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564712+0000", 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4040000000000002e-05 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19375 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3301376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.365 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.807180274, 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049606631999999998, 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19375 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:10.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515271+0000", 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5230000000000001e-06 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515276+0000", 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4650000000000002e-06 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515279+0000", 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0630000000000002e-06 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564738+0000", 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049459001000000002 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564780+0000", 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1693999999999998e-05 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564876+0000", 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5571999999999999e-05 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.369 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19376 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3727360~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.807166362, 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049720590000000002, 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19376 2024-09-16T11:44:10.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517103+0000", 2024-09-16T11:44:10.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001820276 2024-09-16T11:44:10.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517108+0000", 2024-09-16T11:44:10.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5410000000000002e-06 2024-09-16T11:44:10.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517120+0000", 2024-09-16T11:44:10.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2092e-05 2024-09-16T11:44:10.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517125+0000", 2024-09-16T11:44:10.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6829999999999996e-06 2024-09-16T11:44:10.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564870+0000", 2024-09-16T11:44:10.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047745040000000002 2024-09-16T11:44:10.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564921+0000", 2024-09-16T11:44:10.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0538999999999998e-05 2024-09-16T11:44:10.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.374 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.374 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565003+0000", 2024-09-16T11:44:10.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2418999999999998e-05 2024-09-16T11:44:10.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.374 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.374 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19492 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 434176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.374 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.805237505999999, 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048926948999999997, 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19492 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:10.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:10.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3099999999999999e-07 2024-09-16T11:44:10.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517213+0000", 2024-09-16T11:44:10.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6599999999999994e-07 2024-09-16T11:44:10.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517216+0000", 2024-09-16T11:44:10.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.706e-06 2024-09-16T11:44:10.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517218+0000", 2024-09-16T11:44:10.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.035e-06 2024-09-16T11:44:10.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565959+0000", 2024-09-16T11:44:10.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048741380000000001 2024-09-16T11:44:10.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566036+0000", 2024-09-16T11:44:10.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7362999999999998e-05 2024-09-16T11:44:10.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566139+0000", 2024-09-16T11:44:10.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102268 2024-09-16T11:44:10.379 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.379 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.379 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.379 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19493 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 860160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.379 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:10.379 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.805215835, 2024-09-16T11:44:10.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049086604999999998, 2024-09-16T11:44:10.379 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.379 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.379 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.379 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.380 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.380 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19493 2024-09-16T11:44:10.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.380 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:10.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:10.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517234+0000", 2024-09-16T11:44:10.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.37e-07 2024-09-16T11:44:10.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517236+0000", 2024-09-16T11:44:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7150000000000001e-06 2024-09-16T11:44:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517243+0000", 2024-09-16T11:44:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9489999999999999e-06 2024-09-16T11:44:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517246+0000", 2024-09-16T11:44:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.636e-06 2024-09-16T11:44:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566163+0000", 2024-09-16T11:44:10.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048917284999999998 2024-09-16T11:44:10.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566237+0000", 2024-09-16T11:44:10.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3196999999999997e-05 2024-09-16T11:44:10.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566320+0000", 2024-09-16T11:44:10.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3286000000000002e-05 2024-09-16T11:44:10.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.384 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.384 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19494 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1179648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.384 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:10.385 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.805195219, 2024-09-16T11:44:10.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049259562999999999, 2024-09-16T11:44:10.385 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.385 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.385 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.385 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.385 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.385 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19494 2024-09-16T11:44:10.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.385 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:10.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:10.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:10.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:10.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517256+0000", 2024-09-16T11:44:10.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.888e-06 2024-09-16T11:44:10.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517265+0000", 2024-09-16T11:44:10.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8629999999999997e-06 2024-09-16T11:44:10.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517269+0000", 2024-09-16T11:44:10.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2039999999999999e-06 2024-09-16T11:44:10.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566352+0000", 2024-09-16T11:44:10.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049083107000000001 2024-09-16T11:44:10.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566429+0000", 2024-09-16T11:44:10.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7025000000000002e-05 2024-09-16T11:44:10.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.389 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.389 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566514+0000", 2024-09-16T11:44:10.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4121000000000003e-05 2024-09-16T11:44:10.389 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.389 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.389 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.389 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19495 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1286144~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.389 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:10.389 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.805175796, 2024-09-16T11:44:10.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049492880000000003, 2024-09-16T11:44:10.390 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.390 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.390 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.390 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.390 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.390 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19495 2024-09-16T11:44:10.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.390 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517274+0000", 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517276+0000", 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8950000000000001e-06 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517281+0000", 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3879999999999999e-06 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517285+0000", 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5930000000000002e-06 2024-09-16T11:44:10.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566579+0000", 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049294175000000003 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566670+0000", 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1096999999999999e-05 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566766+0000", 2024-09-16T11:44:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6355999999999997e-05 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19496 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1605632~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.805160710999999, 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049632981999999999, 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.395 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19496 2024-09-16T11:44:10.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.395 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:10.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:10.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517289+0000", 2024-09-16T11:44:10.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:10.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517290+0000", 2024-09-16T11:44:10.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.581e-06 2024-09-16T11:44:10.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517300+0000", 2024-09-16T11:44:10.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:10.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517304+0000", 2024-09-16T11:44:10.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:44:10.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566789+0000", 2024-09-16T11:44:10.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049485102000000003 2024-09-16T11:44:10.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566843+0000", 2024-09-16T11:44:10.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3937000000000002e-05 2024-09-16T11:44:10.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566921+0000", 2024-09-16T11:44:10.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8397000000000002e-05 2024-09-16T11:44:10.398 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.398 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.398 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19497 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 114688~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.805141016, 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049761367000000001, 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19497 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519116+0000", 2024-09-16T11:44:10.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018079439999999999 2024-09-16T11:44:10.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519120+0000", 2024-09-16T11:44:10.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2760000000000002e-06 2024-09-16T11:44:10.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519134+0000", 2024-09-16T11:44:10.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3468999999999999e-05 2024-09-16T11:44:10.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519140+0000", 2024-09-16T11:44:10.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.516e-06 2024-09-16T11:44:10.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566943+0000", 2024-09-16T11:44:10.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047802561 2024-09-16T11:44:10.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566982+0000", 2024-09-16T11:44:10.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9269e-05 2024-09-16T11:44:10.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567070+0000", 2024-09-16T11:44:10.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7331999999999997e-05 2024-09-16T11:44:10.403 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.403 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.403 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.404 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19504 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 966656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.404 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:10.404 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.803212529, 2024-09-16T11:44:10.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049045863000000002, 2024-09-16T11:44:10.404 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.404 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.404 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.404 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.404 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.404 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19504 2024-09-16T11:44:10.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.405 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:10.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:10.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:10.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:10.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519239+0000", 2024-09-16T11:44:10.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.962e-06 2024-09-16T11:44:10.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519244+0000", 2024-09-16T11:44:10.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5480000000000001e-06 2024-09-16T11:44:10.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519248+0000", 2024-09-16T11:44:10.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.641e-06 2024-09-16T11:44:10.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568150+0000", 2024-09-16T11:44:10.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048901864000000003 2024-09-16T11:44:10.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568208+0000", 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8399000000000001e-05 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568283+0000", 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4105999999999998e-05 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.408 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19505 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1392640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.803197224, 2024-09-16T11:44:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049202304000000002, 2024-09-16T11:44:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19505 2024-09-16T11:44:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519254+0000", 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.761e-06 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519257+0000", 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.343e-06 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519259+0000", 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0559999999999999e-06 2024-09-16T11:44:10.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568307+0000", 2024-09-16T11:44:10.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049047445000000002 2024-09-16T11:44:10.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568382+0000", 2024-09-16T11:44:10.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5574999999999994e-05 2024-09-16T11:44:10.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568454+0000", 2024-09-16T11:44:10.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1799999999999997e-05 2024-09-16T11:44:10.413 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.413 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.413 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.413 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19506 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1712128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.413 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:10.413 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.803187186000001, 2024-09-16T11:44:10.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049341532, 2024-09-16T11:44:10.413 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.414 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.414 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.414 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.414 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.414 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19506 2024-09-16T11:44:10.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.414 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:10.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521102+0000", 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018404700000000001 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521106+0000", 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8820000000000003e-06 2024-09-16T11:44:10.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521119+0000", 2024-09-16T11:44:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2546e-05 2024-09-16T11:44:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521125+0000", 2024-09-16T11:44:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0249999999999999e-06 2024-09-16T11:44:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568484+0000", 2024-09-16T11:44:10.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047359120999999997 2024-09-16T11:44:10.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568542+0000", 2024-09-16T11:44:10.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8360999999999999e-05 2024-09-16T11:44:10.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568604+0000", 2024-09-16T11:44:10.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1126999999999999e-05 2024-09-16T11:44:10.417 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.418 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.418 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.418 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19515 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1499136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.418 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:10.418 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.801194901000001, 2024-09-16T11:44:10.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048949431000000002, 2024-09-16T11:44:10.418 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.418 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.418 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.418 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.418 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.419 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19515 2024-09-16T11:44:10.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.419 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:10.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:10.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523130+0000", 2024-09-16T11:44:10.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00187604 2024-09-16T11:44:10.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523134+0000", 2024-09-16T11:44:10.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:44:10.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523144+0000", 2024-09-16T11:44:10.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2010000000000005e-06 2024-09-16T11:44:10.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523149+0000", 2024-09-16T11:44:10.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0039999999999999e-06 2024-09-16T11:44:10.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570113+0000", 2024-09-16T11:44:10.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.046964517999999997 2024-09-16T11:44:10.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570157+0000", 2024-09-16T11:44:10.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4056e-05 2024-09-16T11:44:10.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570204+0000", 2024-09-16T11:44:10.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6591999999999997e-05 2024-09-16T11:44:10.422 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.423 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.423 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.423 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.423 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.423 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:10.424 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:10.425 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:10.422+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:10.525 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:10.523+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:10.563 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:10.564 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:10.564 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:10.564 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:10.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:10.626 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:10.624+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:10.672 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:10.672 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:10.672 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:10.672 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:10.672 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:10.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:10.764 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:10.762+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:10.820 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:10.820 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:10.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:10.820 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:10.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.820 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16853 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3375104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.821 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:10.821 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.469661539000001, 2024-09-16T11:44:10.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027791907000000001, 2024-09-16T11:44:10.821 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.821 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.821 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.821 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.821 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.821 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16853 2024-09-16T11:44:10.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.822 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:10.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:10.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347568+0000", 2024-09-16T11:44:10.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:44:10.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347570+0000", 2024-09-16T11:44:10.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:44:10.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347588+0000", 2024-09-16T11:44:10.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7914000000000002e-05 2024-09-16T11:44:10.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347592+0000", 2024-09-16T11:44:10.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3150000000000002e-06 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375199+0000", 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027606065999999999 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375276+0000", 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7527000000000005e-05 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375359+0000", 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3238000000000005e-05 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16854 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.469631842, 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027945148999999999, 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.827 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.828 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.828 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16854 2024-09-16T11:44:10.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.828 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:10.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:10.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347598+0000", 2024-09-16T11:44:10.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:44:10.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347600+0000", 2024-09-16T11:44:10.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:44:10.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347606+0000", 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0580000000000003e-06 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347609+0000", 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375386+0000", 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027776809 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375461+0000", 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4868000000000002e-05 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375542+0000", 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1253000000000003e-05 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.831 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16855 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 4120576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.469615503, 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028111094999999999, 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16855 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:10.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:10.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347614+0000", 2024-09-16T11:44:10.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:44:10.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.834 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347616+0000", 2024-09-16T11:44:10.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:44:10.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347623+0000", 2024-09-16T11:44:10.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0099999999999998e-06 2024-09-16T11:44:10.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347626+0000", 2024-09-16T11:44:10.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2140000000000001e-06 2024-09-16T11:44:10.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375569+0000", 2024-09-16T11:44:10.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027942649999999999 2024-09-16T11:44:10.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375643+0000", 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3949999999999995e-05 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375725+0000", 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1992999999999999e-05 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16856 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3801088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.469593432, 2024-09-16T11:44:10.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028268272000000001, 2024-09-16T11:44:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16856 2024-09-16T11:44:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:10.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347636+0000", 2024-09-16T11:44:10.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:44:10.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347638+0000", 2024-09-16T11:44:10.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.86e-06 2024-09-16T11:44:10.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347644+0000", 2024-09-16T11:44:10.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1739999999999997e-06 2024-09-16T11:44:10.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347656+0000", 2024-09-16T11:44:10.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2198000000000001e-05 2024-09-16T11:44:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375751+0000", 2024-09-16T11:44:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028094507000000001 2024-09-16T11:44:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375824+0000", 2024-09-16T11:44:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3178999999999998e-05 2024-09-16T11:44:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375904+0000", 2024-09-16T11:44:10.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9985000000000001e-05 2024-09-16T11:44:10.843 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.843 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.843 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.843 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16857 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.843 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:10.843 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.469566929999999, 2024-09-16T11:44:10.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028275403000000001, 2024-09-16T11:44:10.843 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.843 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16857 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:10.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:10.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.46e-07 2024-09-16T11:44:10.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347666+0000", 2024-09-16T11:44:10.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8739999999999997e-06 2024-09-16T11:44:10.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347672+0000", 2024-09-16T11:44:10.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1059999999999996e-06 2024-09-16T11:44:10.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347676+0000", 2024-09-16T11:44:10.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:44:10.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375930+0000", 2024-09-16T11:44:10.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028254042 2024-09-16T11:44:10.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375937+0000", 2024-09-16T11:44:10.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5519999999999999e-06 2024-09-16T11:44:10.847 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.847 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.847 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.848 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16858 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.848 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:10.848 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.469549284999999, 2024-09-16T11:44:10.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028284592000000001, 2024-09-16T11:44:10.848 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.848 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:10.848 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.848 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.848 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.848 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16858 2024-09-16T11:44:10.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.849 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:10.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:10.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:10.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:44:10.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347683+0000", 2024-09-16T11:44:10.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6809999999999998e-06 2024-09-16T11:44:10.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347689+0000", 2024-09-16T11:44:10.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:10.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347695+0000", 2024-09-16T11:44:10.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0020000000000003e-06 2024-09-16T11:44:10.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375958+0000", 2024-09-16T11:44:10.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028263870999999999 2024-09-16T11:44:10.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375964+0000", 2024-09-16T11:44:10.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8209999999999998e-06 2024-09-16T11:44:10.854 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.854 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.854 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.855 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16992 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.855 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:10.855 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.467974597, 2024-09-16T11:44:10.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027704776, 2024-09-16T11:44:10.855 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.855 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:10.855 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.855 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.856 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.856 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16992 2024-09-16T11:44:10.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.856 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:10.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:10.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349255+0000", 2024-09-16T11:44:10.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:44:10.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349260+0000", 2024-09-16T11:44:10.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5589999999999996e-06 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349267+0000", 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4019999999999999e-06 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349270+0000", 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6780000000000002e-06 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376953+0000", 2024-09-16T11:44:10.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682933 2024-09-16T11:44:10.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376959+0000", 2024-09-16T11:44:10.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:44:10.860 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.860 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.860 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.860 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16993 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.861 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:10.861 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.467953272000001, 2024-09-16T11:44:10.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027707908999999999, 2024-09-16T11:44:10.861 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.861 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:10.861 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.861 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.861 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.861 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16993 2024-09-16T11:44:10.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.861 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:10.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:10.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:10.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:10.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349278+0000", 2024-09-16T11:44:10.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.629e-06 2024-09-16T11:44:10.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349291+0000", 2024-09-16T11:44:10.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614e-05 2024-09-16T11:44:10.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349295+0000", 2024-09-16T11:44:10.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1189999999999999e-06 2024-09-16T11:44:10.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376978+0000", 2024-09-16T11:44:10.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682445 2024-09-16T11:44:10.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376984+0000", 2024-09-16T11:44:10.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7590000000000003e-06 2024-09-16T11:44:10.865 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.865 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.865 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.865 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16994 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.865 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:10.865 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.467929261, 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027708753999999999, 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16994 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:44:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349302+0000", 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.722e-06 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349308+0000", 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349312+0000", 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5310000000000002e-06 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377003+0000", 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027691308000000001 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377008+0000", 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4550000000000003e-06 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16999 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.869 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.467786383, 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027705270000000001, 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16999 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:10.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.46e-07 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.871 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349445+0000", 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.637e-06 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349451+0000", 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9959999999999999e-06 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349454+0000", 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.647e-06 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.872 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377142+0000", 2024-09-16T11:44:10.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027687942 2024-09-16T11:44:10.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377148+0000", 2024-09-16T11:44:10.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7019999999999998e-06 2024-09-16T11:44:10.873 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.873 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.873 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.873 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17000 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.467770875999999, 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.02771881, 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17000 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0600000000000001e-07 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.876 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349460+0000", 2024-09-16T11:44:10.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.736e-06 2024-09-16T11:44:10.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.876 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.876 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349466+0000", 2024-09-16T11:44:10.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2419999999999997e-06 2024-09-16T11:44:10.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.876 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.876 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349470+0000", 2024-09-16T11:44:10.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5480000000000002e-06 2024-09-16T11:44:10.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.877 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377171+0000", 2024-09-16T11:44:10.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027701282000000001 2024-09-16T11:44:10.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.877 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377177+0000", 2024-09-16T11:44:10.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5960000000000003e-06 2024-09-16T11:44:10.877 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.877 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.877 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17001 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.467755126, 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027739243, 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17001 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:44:10.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349476+0000", 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349482+0000", 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2389999999999999e-06 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349486+0000", 2024-09-16T11:44:10.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5980000000000001e-06 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377206+0000", 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027720037999999999 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377213+0000", 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3440000000000003e-06 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17433 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 5898240~4888] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.431791472, 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027709398, 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17433 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385438+0000", 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:10.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385439+0000", 2024-09-16T11:44:10.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5120000000000001e-06 2024-09-16T11:44:10.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385445+0000", 2024-09-16T11:44:10.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4e-06 2024-09-16T11:44:10.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385448+0000", 2024-09-16T11:44:10.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1669999999999999e-06 2024-09-16T11:44:10.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412916+0000", 2024-09-16T11:44:10.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027468137 2024-09-16T11:44:10.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412988+0000", 2024-09-16T11:44:10.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1606000000000005e-05 2024-09-16T11:44:10.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413147+0000", 2024-09-16T11:44:10.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015924299999999999 2024-09-16T11:44:10.887 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.888 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.888 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.888 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17434 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6004736~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.889 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:10.889 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.431776881999999, 2024-09-16T11:44:10.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027777461, 2024-09-16T11:44:10.889 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.889 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.889 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.890 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.890 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.890 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17434 2024-09-16T11:44:10.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.890 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.890 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:10.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:10.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:10.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:10.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385454+0000", 2024-09-16T11:44:10.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6729999999999999e-06 2024-09-16T11:44:10.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385459+0000", 2024-09-16T11:44:10.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4079999999999997e-06 2024-09-16T11:44:10.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385463+0000", 2024-09-16T11:44:10.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.269e-06 2024-09-16T11:44:10.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413157+0000", 2024-09-16T11:44:10.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027694611000000001 2024-09-16T11:44:10.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413193+0000", 2024-09-16T11:44:10.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5673999999999998e-05 2024-09-16T11:44:10.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413230+0000", 2024-09-16T11:44:10.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6454e-05 2024-09-16T11:44:10.895 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.895 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.895 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.895 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17435 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6324224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.895 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:10.895 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.431762729000001, 2024-09-16T11:44:10.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929059999999999, 2024-09-16T11:44:10.895 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.895 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.895 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.896 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17435 2024-09-16T11:44:10.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.896 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.896 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.896 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:10.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.896 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.896 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:10.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385467+0000", 2024-09-16T11:44:10.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:10.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385468+0000", 2024-09-16T11:44:10.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6300000000000001e-06 2024-09-16T11:44:10.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385474+0000", 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1299999999999998e-06 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385478+0000", 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2880000000000001e-06 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413257+0000", 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027779608000000001 2024-09-16T11:44:10.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413331+0000", 2024-09-16T11:44:10.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3639000000000004e-05 2024-09-16T11:44:10.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413395+0000", 2024-09-16T11:44:10.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4424999999999994e-05 2024-09-16T11:44:10.900 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.900 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.900 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.900 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17436 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7176192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.900 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:10.900 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.431747067, 2024-09-16T11:44:10.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028016718999999999, 2024-09-16T11:44:10.900 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.900 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.900 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.901 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.901 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.901 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17436 2024-09-16T11:44:10.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.901 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:10.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:10.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:10.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:10.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385484+0000", 2024-09-16T11:44:10.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5179999999999999e-06 2024-09-16T11:44:10.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385489+0000", 2024-09-16T11:44:10.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7049999999999996e-06 2024-09-16T11:44:10.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385493+0000", 2024-09-16T11:44:10.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T11:44:10.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413422+0000", 2024-09-16T11:44:10.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929611999999999 2024-09-16T11:44:10.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413462+0000", 2024-09-16T11:44:10.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9585000000000001e-05 2024-09-16T11:44:10.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413499+0000", 2024-09-16T11:44:10.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6644999999999999e-05 2024-09-16T11:44:10.907 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.907 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.907 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.908 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17437 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6750208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.908 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:10.908 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.431732652000001, 2024-09-16T11:44:10.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028167168999999999, 2024-09-16T11:44:10.908 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.909 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.909 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17437 2024-09-16T11:44:10.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.910 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:10.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:10.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385497+0000", 2024-09-16T11:44:10.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:44:10.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385498+0000", 2024-09-16T11:44:10.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.592e-06 2024-09-16T11:44:10.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385504+0000", 2024-09-16T11:44:10.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4090000000000002e-06 2024-09-16T11:44:10.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.918 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385513+0000", 2024-09-16T11:44:10.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4460000000000007e-06 2024-09-16T11:44:10.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.918 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.918 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413526+0000", 2024-09-16T11:44:10.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028012866000000001 2024-09-16T11:44:10.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413601+0000", 2024-09-16T11:44:10.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5388000000000004e-05 2024-09-16T11:44:10.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413663+0000", 2024-09-16T11:44:10.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.211e-05 2024-09-16T11:44:10.921 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.921 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.921 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.921 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17438 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7495680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.922 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:10.922 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.431709893000001, 2024-09-16T11:44:10.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028224439, 2024-09-16T11:44:10.922 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.922 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.923 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.923 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.923 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.923 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17438 2024-09-16T11:44:10.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.923 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:10.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:10.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:10.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4900000000000001e-07 2024-09-16T11:44:10.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385521+0000", 2024-09-16T11:44:10.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:44:10.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385526+0000", 2024-09-16T11:44:10.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2680000000000002e-06 2024-09-16T11:44:10.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385529+0000", 2024-09-16T11:44:10.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2040000000000002e-06 2024-09-16T11:44:10.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413674+0000", 2024-09-16T11:44:10.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028144226000000001 2024-09-16T11:44:10.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413710+0000", 2024-09-16T11:44:10.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6665000000000002e-05 2024-09-16T11:44:10.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413743+0000", 2024-09-16T11:44:10.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3155999999999999e-05 2024-09-16T11:44:10.931 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.931 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.931 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.932 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17439 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7921664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.932 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:10.932 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.431648739, 2024-09-16T11:44:10.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028326131000000001, 2024-09-16T11:44:10.932 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.933 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.933 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.933 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.933 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.933 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17439 2024-09-16T11:44:10.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.934 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:10.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:10.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385581+0000", 2024-09-16T11:44:10.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9299999999999998e-07 2024-09-16T11:44:10.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385583+0000", 2024-09-16T11:44:10.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6220000000000001e-06 2024-09-16T11:44:10.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385592+0000", 2024-09-16T11:44:10.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5220000000000007e-06 2024-09-16T11:44:10.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385595+0000", 2024-09-16T11:44:10.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8669999999999998e-06 2024-09-16T11:44:10.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413771+0000", 2024-09-16T11:44:10.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028175875999999999 2024-09-16T11:44:10.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413845+0000", 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4382999999999996e-05 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413906+0000", 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1367999999999999e-05 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17440 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.431631531000001, 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028391117, 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17440 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:10.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:10.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:10.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:10.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:10.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:10.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385598+0000", 2024-09-16T11:44:10.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:44:10.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385603+0000", 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385607+0000", 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4869999999999998e-06 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385609+0000", 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2759999999999999e-06 2024-09-16T11:44:10.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:10.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413916+0000", 2024-09-16T11:44:10.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028307253000000001 2024-09-16T11:44:10.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:10.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413955+0000", 2024-09-16T11:44:10.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8652000000000002e-05 2024-09-16T11:44:10.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:10.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:10.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:10.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413989+0000", 2024-09-16T11:44:10.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3587999999999998e-05 2024-09-16T11:44:10.942 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.942 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.942 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.942 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:10.942 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:10.943 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:10.944 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:10.945 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:10.941+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:10.995 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-16T11:44:11.045 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:11.042+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:11.097 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:11.097 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:11.098 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:11.098 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:11.107 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:11.146 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:11.144+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:11.243 INFO:teuthology.orchestra.run.smithi070.stdout:4017 trunc from 0xaf5a19b to 0x3d5384a 2024-09-16T11:44:11.247 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:11.245+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:11.251 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:11.251 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:11.267 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:11.302 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:44:11.357 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:11.355+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:11.367 INFO:teuthology.orchestra.run.smithi070.stdout:4018 punch from 0x37305bf to 0x373c058, (0xba99 bytes) 2024-09-16T11:44:11.368 INFO:teuthology.orchestra.run.smithi070.stdout:4020 read 0x3a67584 thru 0x3a763fd (0xee7a bytes) 2024-09-16T11:44:11.371 INFO:teuthology.orchestra.run.smithi070.stdout:4021 trunc from 0x3d5384a to 0x151eb23 2024-09-16T11:44:11.379 INFO:teuthology.orchestra.run.smithi070.stdout:4022 read 0xdfa496 thru 0xe06900 (0xc46b bytes) 2024-09-16T11:44:11.387 INFO:teuthology.orchestra.run.smithi070.stdout:4023 write 0x2d2de02 thru 0x2d3a363 (0xc562 bytes) 2024-09-16T11:44:11.389 INFO:teuthology.orchestra.run.smithi070.stdout:4025 read 0x285ba3d thru 0x2869e65 (0xe429 bytes) 2024-09-16T11:44:11.389 INFO:teuthology.orchestra.run.smithi070.stdout:4026 read 0x941e06 thru 0x951c27 (0xfe22 bytes) 2024-09-16T11:44:11.403 INFO:teuthology.orchestra.run.smithi070.stdout:4028 write 0x9d1de64 thru 0x9d2b4ac (0xd649 bytes) 2024-09-16T11:44:11.407 INFO:teuthology.orchestra.run.smithi070.stdout:4029 read 0x82d73d3 thru 0x82dc8d3 (0x5501 bytes) 2024-09-16T11:44:11.407 INFO:teuthology.orchestra.run.smithi070.stdout:4031 read 0x451ff02 thru 0x452a403 (0xa502 bytes) 2024-09-16T11:44:11.407 INFO:teuthology.orchestra.run.smithi070.stdout:4032 clone 25 order 21 su 16384 sc 3 2024-09-16T11:44:11.415 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:11.415 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:11.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:11.415 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:11.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.415 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19545 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4415488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.415 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.888738718000001, 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043429581000000002, 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19545 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:11.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:11.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:11.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:11.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523839+0000", 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0220000000000004e-06 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523845+0000", 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6500000000000001e-06 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523848+0000", 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1879999999999998e-06 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567092+0000", 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043244145999999997 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567176+0000", 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3825999999999993e-05 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567263+0000", 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7423999999999995e-05 2024-09-16T11:44:11.419 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.420 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.420 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.420 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19546 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4734976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.420 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:11.420 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.888720878000001, 2024-09-16T11:44:11.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043599231000000002, 2024-09-16T11:44:11.420 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.420 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.420 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.420 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.420 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.421 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19546 2024-09-16T11:44:11.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.421 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:11.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523954+0000", 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010217099999999999 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523957+0000", 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3270000000000001e-06 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523966+0000", 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6489999999999997e-06 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523971+0000", 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567291+0000", 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043319931999999998 2024-09-16T11:44:11.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567360+0000", 2024-09-16T11:44:11.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8628999999999996e-05 2024-09-16T11:44:11.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567451+0000", 2024-09-16T11:44:11.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1012000000000002e-05 2024-09-16T11:44:11.424 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.424 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.424 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19547 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4841472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.888596021, 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043658858000000002, 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19547 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525102+0000", 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011257730000000001 2024-09-16T11:44:11.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525106+0000", 2024-09-16T11:44:11.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9910000000000004e-06 2024-09-16T11:44:11.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525116+0000", 2024-09-16T11:44:11.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0231000000000001e-05 2024-09-16T11:44:11.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525137+0000", 2024-09-16T11:44:11.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0828000000000001e-05 2024-09-16T11:44:11.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567478+0000", 2024-09-16T11:44:11.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.042340467999999999 2024-09-16T11:44:11.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567554+0000", 2024-09-16T11:44:11.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6074000000000004e-05 2024-09-16T11:44:11.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567635+0000", 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1493000000000001e-05 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19553 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4947968~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.887368959, 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043450397000000002, 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.429 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19553 2024-09-16T11:44:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1799999999999999e-07 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525205+0000", 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.062e-06 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525208+0000", 2024-09-16T11:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8739999999999999e-06 2024-09-16T11:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525209+0000", 2024-09-16T11:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.725e-06 2024-09-16T11:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568520+0000", 2024-09-16T11:44:11.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043310226 2024-09-16T11:44:11.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568569+0000", 2024-09-16T11:44:11.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8959999999999999e-05 2024-09-16T11:44:11.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568654+0000", 2024-09-16T11:44:11.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5332000000000002e-05 2024-09-16T11:44:11.433 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.433 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.433 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19554 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5267456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.887359967, 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043659587, 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19554 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.434 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:11.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:11.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527103+0000", 2024-09-16T11:44:11.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018907419999999999 2024-09-16T11:44:11.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527108+0000", 2024-09-16T11:44:11.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6040000000000003e-06 2024-09-16T11:44:11.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527118+0000", 2024-09-16T11:44:11.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527122+0000", 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9559999999999996e-06 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568688+0000", 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041565116999999999 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568768+0000", 2024-09-16T11:44:11.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0474000000000002e-05 2024-09-16T11:44:11.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568872+0000", 2024-09-16T11:44:11.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104028 2024-09-16T11:44:11.438 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.438 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.438 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.438 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19565 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6332416~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.438 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:11.439 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.885281395, 2024-09-16T11:44:11.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043494890000000001, 2024-09-16T11:44:11.439 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.439 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.439 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.439 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.439 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.439 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19565 2024-09-16T11:44:11.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.439 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527292+0000", 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.27e-07 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527293+0000", 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.917e-06 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527299+0000", 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6699999999999999e-06 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527304+0000", 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6160000000000004e-06 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570683+0000", 2024-09-16T11:44:11.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043379171000000001 2024-09-16T11:44:11.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570733+0000", 2024-09-16T11:44:11.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0485000000000002e-05 2024-09-16T11:44:11.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570786+0000", 2024-09-16T11:44:11.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2604e-05 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19566 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6758400~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.885265061, 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043643183000000002, 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.444 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19566 2024-09-16T11:44:11.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.444 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:11.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:11.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527308+0000", 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527310+0000", 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.68e-06 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527316+0000", 2024-09-16T11:44:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9730000000000002e-06 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527319+0000", 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.027e-06 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570810+0000", 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043491525000000003 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570882+0000", 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570951+0000", 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8782000000000007e-05 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19567 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:11.447 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.885250355, 2024-09-16T11:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043786291999999997, 2024-09-16T11:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19567 2024-09-16T11:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:11.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529108+0000", 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001785656 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529113+0000", 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1080000000000001e-06 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529122+0000", 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0550000000000005e-06 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529127+0000", 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9849999999999997e-06 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570997+0000", 2024-09-16T11:44:11.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041869811 2024-09-16T11:44:11.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571063+0000", 2024-09-16T11:44:11.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6340999999999994e-05 2024-09-16T11:44:11.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571108+0000", 2024-09-16T11:44:11.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5336000000000001e-05 2024-09-16T11:44:11.451 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19580 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 8036352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.883275054, 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043451596000000002, 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.452 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19580 2024-09-16T11:44:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531106+0000", 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001808179 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531110+0000", 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5990000000000004e-06 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531123+0000", 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2687000000000001e-05 2024-09-16T11:44:11.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531129+0000", 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572642+0000", 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041513474000000002 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572691+0000", 2024-09-16T11:44:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8989000000000001e-05 2024-09-16T11:44:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572749+0000", 2024-09-16T11:44:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7806999999999997e-05 2024-09-16T11:44:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19708 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4530176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.881228489, 2024-09-16T11:44:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394863999999998, 2024-09-16T11:44:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19708 2024-09-16T11:44:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.67e-07 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531346+0000", 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.477e-06 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531351+0000", 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1050000000000003e-06 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531354+0000", 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9960000000000002e-06 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574620+0000", 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043266365000000001 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574662+0000", 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1662000000000002e-05 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574739+0000", 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6991999999999997e-05 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.460 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19709 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4849664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.881215013, 2024-09-16T11:44:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043537291999999998, 2024-09-16T11:44:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.461 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19709 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531358+0000", 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531360+0000", 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531365+0000", 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.716e-06 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531369+0000", 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5410000000000001e-06 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574763+0000", 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394359 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574831+0000", 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7997999999999995e-05 2024-09-16T11:44:11.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574895+0000", 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3427000000000001e-05 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19710 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4956160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.881199733999999, 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043674198999999997, 2024-09-16T11:44:11.465 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.466 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.466 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.466 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.466 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.466 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19710 2024-09-16T11:44:11.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.466 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.466 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:11.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533105+0000", 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001731961 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533109+0000", 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3080000000000001e-06 2024-09-16T11:44:11.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533120+0000", 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0625000000000001e-05 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533125+0000", 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574923+0000", 2024-09-16T11:44:11.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041797187 2024-09-16T11:44:11.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574991+0000", 2024-09-16T11:44:11.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8752999999999997e-05 2024-09-16T11:44:11.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.575047+0000", 2024-09-16T11:44:11.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5596000000000001e-05 2024-09-16T11:44:11.469 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.469 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.469 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19719 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5488640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.879227985, 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043412013999999999, 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19719 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.471 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:11.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.471 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:11.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.471 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533345+0000", 2024-09-16T11:44:11.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:44:11.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533347+0000", 2024-09-16T11:44:11.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8139999999999999e-06 2024-09-16T11:44:11.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533352+0000", 2024-09-16T11:44:11.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4970000000000001e-06 2024-09-16T11:44:11.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533355+0000", 2024-09-16T11:44:11.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.711e-06 2024-09-16T11:44:11.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576629+0000", 2024-09-16T11:44:11.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043274295999999997 2024-09-16T11:44:11.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576706+0000", 2024-09-16T11:44:11.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6711000000000005e-05 2024-09-16T11:44:11.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576757+0000", 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0509e-05 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19720 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5808128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.879215336, 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043577006000000001, 2024-09-16T11:44:11.474 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19720 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.35e-07 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533359+0000", 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533364+0000", 2024-09-16T11:44:11.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4349999999999996e-06 2024-09-16T11:44:11.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533367+0000", 2024-09-16T11:44:11.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2349999999999999e-06 2024-09-16T11:44:11.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576786+0000", 2024-09-16T11:44:11.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043418885999999997 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576862+0000", 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6172000000000002e-05 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576934+0000", 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1891999999999995e-05 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19721 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6234112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.879201816, 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043681656999999999, 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19721 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:11.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:11.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:11.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-07 2024-09-16T11:44:11.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533373+0000", 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.973e-06 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533377+0000", 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.403e-06 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533379+0000", 2024-09-16T11:44:11.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.058e-06 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576959+0000", 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043579433000000001 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577015+0000", 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5859000000000002e-05 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577052+0000", 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7620999999999998e-05 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19722 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6340608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.879191089000001, 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043854922999999997, 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.483 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.484 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.484 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19722 2024-09-16T11:44:11.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.484 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:11.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:11.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535103+0000", 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017217020000000001 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535107+0000", 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2819999999999998e-06 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535118+0000", 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0148e-05 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535123+0000", 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577091+0000", 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041967930000000001 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577163+0000", 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1853999999999993e-05 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577236+0000", 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3699e-05 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19732 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6447104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.487 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:11.488 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.877331233, 2024-09-16T11:44:11.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043399048000000003, 2024-09-16T11:44:11.488 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.488 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.488 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.488 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.488 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.488 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19732 2024-09-16T11:44:11.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.488 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:11.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:11.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535242+0000", 2024-09-16T11:44:11.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:11.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535243+0000", 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.793e-06 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535249+0000", 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.643e-06 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535252+0000", 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:44:11.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578501+0000", 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043248343000000002 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578576+0000", 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5544999999999996e-05 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578640+0000", 2024-09-16T11:44:11.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4018999999999998e-05 2024-09-16T11:44:11.492 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.492 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.492 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.492 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19733 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6873088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.492 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:11.492 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.877310392, 2024-09-16T11:44:11.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043549681, 2024-09-16T11:44:11.492 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.492 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.492 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.493 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.493 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.493 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19733 2024-09-16T11:44:11.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.493 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:11.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535263+0000", 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535265+0000", 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9369999999999998e-06 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535271+0000", 2024-09-16T11:44:11.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9719999999999997e-06 2024-09-16T11:44:11.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535274+0000", 2024-09-16T11:44:11.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:44:11.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578668+0000", 2024-09-16T11:44:11.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043393355000000002 2024-09-16T11:44:11.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578744+0000", 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6853000000000004e-05 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578812+0000", 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.737e-05 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.497 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19734 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7299072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.497 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:11.497 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.877294644999999, 2024-09-16T11:44:11.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043699189999999999, 2024-09-16T11:44:11.497 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.497 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.497 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.497 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.497 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.497 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19734 2024-09-16T11:44:11.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.497 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:11.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:11.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537101+0000", 2024-09-16T11:44:11.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001823098 2024-09-16T11:44:11.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537105+0000", 2024-09-16T11:44:11.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:11.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537119+0000", 2024-09-16T11:44:11.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614999999999999e-05 2024-09-16T11:44:11.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537125+0000", 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578838+0000", 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041712605999999999 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578912+0000", 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4396999999999999e-05 2024-09-16T11:44:11.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578977+0000", 2024-09-16T11:44:11.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4858999999999997e-05 2024-09-16T11:44:11.501 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.501 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.501 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.501 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19851 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4431872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:11.501 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:11.501 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.875213901, 2024-09-16T11:44:11.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043564540999999998, 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19851 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539100+0000", 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017412079999999999 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539105+0000", 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T11:44:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539116+0000", 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1824999999999999e-05 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539122+0000", 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9859999999999999e-06 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580764+0000", 2024-09-16T11:44:11.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041641299 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580839+0000", 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5167999999999996e-05 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580923+0000", 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4314999999999995e-05 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.505 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:11.506 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:11.506 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:11.507 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:11.507 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:11.508 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:11.508 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:11.508 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:11.509 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:11.509 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:11.509 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:11.509 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:11.510 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:11.505+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:11.520 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:11.520 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:11.535 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:11.536 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:11.554 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:11.554 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:11.571 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:11.571 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:11.608 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:11.605+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:11.613 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:11.613 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:11.621 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:11.709 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:11.707+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:11.760 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:11.760 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:11.768 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:11.810 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:11.808+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:11.865 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:11.866 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:11.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:11.918 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:11.916+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:11.929 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:44:11.929 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:44:11.929 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T11:44:11.929 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-16T11:44:11.930 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:44:11.977 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:11.977 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:11.981 INFO:teuthology.orchestra.run.smithi070.stdout:flattening image image_client.0-clone24 2024-09-16T11:44:11.985 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:12.026 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:12.024+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:12.105 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-16T11:44:12.112 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:12.110+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:12.120 INFO:teuthology.orchestra.run.smithi070.stdout:4033 punch from 0x535ae0e to 0x5362713, (0x7905 bytes) 2024-09-16T11:44:12.125 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:12.125 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:12.134 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:12.134 INFO:teuthology.orchestra.run.smithi070.stdout:4034 write 0xd3bdce2 thru 0xd3ca30f (0xc62e bytes) 2024-09-16T11:44:12.138 INFO:teuthology.orchestra.run.smithi070.stdout:4037 read 0x1e53fda thru 0x1e5434c (0x373 bytes) 2024-09-16T11:44:12.139 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.139 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.139 INFO:teuthology.orchestra.run.smithi070.stdout:4042 write 0x9f8601c thru 0x9f87a69 (0x1a4e bytes) 2024-09-16T11:44:12.160 INFO:teuthology.orchestra.run.smithi070.stdout:4043 trunc from 0xd3ca310 to 0x3f0ca6 2024-09-16T11:44:12.205 INFO:teuthology.orchestra.run.smithi070.stdout:4044 writesame 0x269ece1 thru 0x26a8363 data_size 0x1(0x9683 bytes) 2024-09-16T11:44:12.213 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:12.211+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:12.223 INFO:teuthology.orchestra.run.smithi070.stdout:4045 write 0xc124c73 thru 0xc128514 (0x38a2 bytes) 2024-09-16T11:44:12.227 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.227 INFO:teuthology.orchestra.run.smithi070.stdout:4048 writesame 0x7320617 thru 0x7326c4e data_size 0x4(0x6638 bytes) 2024-09-16T11:44:12.232 INFO:teuthology.orchestra.run.smithi070.stdout:4049 writesame 0x97224b5 thru 0x9726890 data_size 0x56(0x43dc bytes) 2024-09-16T11:44:12.236 INFO:teuthology.orchestra.run.smithi070.stdout:4050 write 0x9146bc2 thru 0x914fda4 (0x91e3 bytes) 2024-09-16T11:44:12.237 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:12.237 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:12.241 INFO:teuthology.orchestra.run.smithi070.stdout:4052 writesame 0x9559eda thru 0x95689cf data_size 0x4b(0xeaf6 bytes) 2024-09-16T11:44:12.244 INFO:teuthology.orchestra.run.smithi070.stdout:4053 trunc from 0xc128515 to 0xc1a9705 2024-09-16T11:44:12.245 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:44:12.245 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:44:12.246 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.52668 -> 3.52668 (expected baseline 3.13748) 2024-09-16T11:44:12.246 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:44:12.246 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 19 -> 19 pgs (1.68889 -> 1.68889 * mean) 2024-09-16T11:44:12.246 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:12.246 INFO:teuthology.orchestra.run.smithi070.stdout:oload 110 2024-09-16T11:44:12.246 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 3 2024-09-16T11:44:12.246 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:44:12.246 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0033 2024-09-16T11:44:12.246 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0036 2024-09-16T11:44:12.246 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:44:12.247 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:12.249 INFO:teuthology.orchestra.run.smithi070.stdout:4054 writesame 0xb8d782d thru 0xb8d9e63 data_size 0x3(0x2637 bytes) 2024-09-16T11:44:12.254 INFO:teuthology.orchestra.run.smithi070.stdout:4055 read 0x5f66229 thru 0x5f6e105 (0x7edd bytes) 2024-09-16T11:44:12.254 INFO:teuthology.orchestra.run.smithi070.stdout:4056 punch from 0x936e732 to 0x937433e, (0x5c0c bytes) 2024-09-16T11:44:12.260 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-16T11:44:12.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-16T11:44:12.261 INFO:teuthology.orchestra.run.smithi070.stdout:4057 write 0xcbbaef3 thru 0xcbbd380 (0x248e bytes) 2024-09-16T11:44:12.265 INFO:teuthology.orchestra.run.smithi070.stdout:4058 read 0xb873f66 thru 0xb8745bd (0x658 bytes) 2024-09-16T11:44:12.265 INFO:teuthology.orchestra.run.smithi070.stdout:4059 write 0x4c96dc4 thru 0x4ca03dc (0x9619 bytes) 2024-09-16T11:44:12.269 INFO:teuthology.orchestra.run.smithi070.stdout:4060 write 0xe18257 thru 0xe256e1 (0xd48b bytes) 2024-09-16T11:44:12.275 INFO:teuthology.orchestra.run.smithi070.stdout:4061 punch from 0x763ab2a to 0x76489c3, (0xde99 bytes) 2024-09-16T11:44:12.281 INFO:teuthology.orchestra.run.smithi070.stdout:4062 trunc from 0xcbbd381 to 0x71ce321 2024-09-16T11:44:12.292 INFO:teuthology.orchestra.run.smithi070.stdout:4065 write 0x3241c1e thru 0x3242ec9 (0x12ac bytes) 2024-09-16T11:44:12.308 INFO:teuthology.orchestra.run.smithi070.stdout:4067 writesame 0xdeef450 thru 0xdef82ac data_size 0x25(0x8e5d bytes) 2024-09-16T11:44:12.318 INFO:teuthology.orchestra.run.smithi070.stdout:4068 trunc from 0xdef82ad to 0x59a8a89 2024-09-16T11:44:12.323 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.323 INFO:teuthology.orchestra.run.smithi070.stdout:4070 read 0x1737e14 thru 0x173854f (0x73c bytes) 2024-09-16T11:44:12.323 INFO:teuthology.orchestra.run.smithi070.stdout:4071 punch from 0x48b3333 to 0x48bf849, (0xc516 bytes) 2024-09-16T11:44:12.331 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:12.329+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:12.335 INFO:teuthology.orchestra.run.smithi070.stdout:4072 write 0xbfd7b96 thru 0xbfe0987 (0x8df2 bytes) 2024-09-16T11:44:12.340 INFO:teuthology.orchestra.run.smithi070.stdout:4073 punch from 0x6a9d967 to 0x6aa4502, (0x6b9b bytes) 2024-09-16T11:44:12.340 INFO:teuthology.orchestra.run.smithi070.stdout:4074 punch from 0x73c51cd to 0x73cbc6e, (0x6aa1 bytes) 2024-09-16T11:44:12.340 INFO:teuthology.orchestra.run.smithi070.stdout:4075 read 0xac734e4 thru 0xac78453 (0x4f70 bytes) 2024-09-16T11:44:12.341 INFO:teuthology.orchestra.run.smithi070.stdout:4076 read 0x5c14e8 thru 0x5d0d75 (0xf88e bytes) 2024-09-16T11:44:12.345 INFO:teuthology.orchestra.run.smithi070.stdout:4077 trunc from 0xbfe0988 to 0x7fc51ff 2024-09-16T11:44:12.355 INFO:teuthology.orchestra.run.smithi070.stdout:4078 trunc from 0x7fc51ff to 0x7ae2595 2024-09-16T11:44:12.359 INFO:teuthology.orchestra.run.smithi070.stdout:4079 read 0x2ea4d7a thru 0x2ea66b8 (0x193f bytes) 2024-09-16T11:44:12.359 INFO:teuthology.orchestra.run.smithi070.stdout:4081 write 0x63bd51a thru 0x63c189b (0x4382 bytes) 2024-09-16T11:44:12.364 INFO:teuthology.orchestra.run.smithi070.stdout:4082 write 0x751d871 thru 0x752b8e9 (0xe079 bytes) 2024-09-16T11:44:12.368 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.368 INFO:teuthology.orchestra.run.smithi070.stdout:4085 write 0x4ca68fe thru 0x4cb47eb (0xdeee bytes) 2024-09-16T11:44:12.369 INFO:teuthology.orchestra.run.smithi070.stdout:4087 write 0x262f816 thru 0x263ee54 (0xf63f bytes) 2024-09-16T11:44:12.376 INFO:teuthology.orchestra.run.smithi070.stdout:4088 write 0x2b821fe thru 0x2b86027 (0x3e2a bytes) 2024-09-16T11:44:12.379 INFO:teuthology.orchestra.run.smithi070.stdout:4090 read 0x4961cc6 thru 0x4964f44 (0x327f bytes) 2024-09-16T11:44:12.379 INFO:teuthology.orchestra.run.smithi070.stdout:4091 write 0x10fef4b thru 0x1103c6f (0x4d25 bytes) 2024-09-16T11:44:12.381 INFO:teuthology.orchestra.run.smithi070.stdout:4092 writesame 0x592247d thru 0x5930c33 data_size 0x75(0xe7b7 bytes) 2024-09-16T11:44:12.386 INFO:teuthology.orchestra.run.smithi070.stdout:4094 punch from 0x631920a to 0x631bd7e, (0x2b74 bytes) 2024-09-16T11:44:12.386 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.386 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.389 INFO:teuthology.orchestra.run.smithi070.stdout:4097 trunc from 0x7ae2595 to 0xa082c4c 2024-09-16T11:44:12.389 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:12.389 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:12.398 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:12.399 INFO:teuthology.orchestra.run.smithi070.stdout:4099 trunc from 0xa082c4c to 0xe8af7c3 2024-09-16T11:44:12.404 INFO:teuthology.orchestra.run.smithi070.stdout:4100 write 0xa5f1efe thru 0xa5ff7d9 (0xd8dc bytes) 2024-09-16T11:44:12.409 INFO:teuthology.orchestra.run.smithi070.stdout:4101 punch from 0x2429fa0 to 0x2434d0f, (0xad6f bytes) 2024-09-16T11:44:12.409 INFO:teuthology.orchestra.run.smithi070.stdout:4102 punch from 0xe554b2d to 0xe55a4b8, (0x598b bytes) 2024-09-16T11:44:12.409 INFO:teuthology.orchestra.run.smithi070.stdout:4103 write 0x764c9da thru 0x7650ba5 (0x41cc bytes) 2024-09-16T11:44:12.410 INFO:teuthology.orchestra.run.smithi070.stdout:4104 writesame 0x589f1a3 thru 0x58acebf data_size 0x1(0xdd1d bytes) 2024-09-16T11:44:12.414 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:12.412+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:12.436 INFO:teuthology.orchestra.run.smithi070.stdout:4105 trunc from 0xe8af7c3 to 0x31f6f33 2024-09-16T11:44:12.448 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.459 INFO:teuthology.orchestra.run.smithi070.stdout:4107 trunc from 0x31f6f33 to 0xc403464 2024-09-16T11:44:12.466 INFO:teuthology.orchestra.run.smithi070.stdout:4109 trunc from 0xc403464 to 0xa73d1ce 2024-09-16T11:44:12.471 INFO:teuthology.orchestra.run.smithi070.stdout:4110 write 0x6d5d692 thru 0x6d5ddeb (0x75a bytes) 2024-09-16T11:44:12.474 INFO:teuthology.orchestra.run.smithi070.stdout:4111 write 0x66b49dd thru 0x66be246 (0x986a bytes) 2024-09-16T11:44:12.478 INFO:teuthology.orchestra.run.smithi070.stdout:4113 read 0x320b6c5 thru 0x321105c (0x5998 bytes) 2024-09-16T11:44:12.489 INFO:teuthology.orchestra.run.smithi070.stdout:4114 trunc from 0xa73d1ce to 0x1b7950b 2024-09-16T11:44:12.511 INFO:teuthology.orchestra.run.smithi070.stdout:4115 write 0x870921f thru 0x8718897 (0xf679 bytes) 2024-09-16T11:44:12.515 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:12.513+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:12.516 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.523 INFO:teuthology.orchestra.run.smithi070.stdout:4118 write 0xa1c4d3c thru 0xa1d32c4 (0xe589 bytes) 2024-09-16T11:44:12.527 INFO:teuthology.orchestra.run.smithi070.stdout:4119 writesame 0x6ed1de0 thru 0x6edc20e data_size 0x1(0xa42f bytes) 2024-09-16T11:44:12.539 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:12.539 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:12.547 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:12.551 INFO:teuthology.orchestra.run.smithi070.stdout:4120 write 0xda27a29 thru 0xda33a63 (0xc03b bytes) 2024-09-16T11:44:12.556 INFO:teuthology.orchestra.run.smithi070.stdout:4121 punch from 0x9b6df4a to 0x9b6e440, (0x4f6 bytes) 2024-09-16T11:44:12.556 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.556 INFO:teuthology.orchestra.run.smithi070.stdout:4123 writesame 0x9ab18b7 thru 0x9abafd7 data_size 0x1(0x9721 bytes) 2024-09-16T11:44:12.582 INFO:teuthology.orchestra.run.smithi070.stdout:4126 trunc from 0xda33a64 to 0x47d2aa4 2024-09-16T11:44:12.588 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:44:12.588 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:44:12.588 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.52668 -> 3.52668 (expected baseline 3.13748) 2024-09-16T11:44:12.588 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:44:12.588 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 19 -> 19 pgs (1.68889 -> 1.68889 * mean) 2024-09-16T11:44:12.588 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:12.588 INFO:teuthology.orchestra.run.smithi070.stdout:oload 1000 2024-09-16T11:44:12.589 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 0.05 2024-09-16T11:44:12.589 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:44:12.589 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0033 2024-09-16T11:44:12.589 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0327 2024-09-16T11:44:12.589 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:44:12.593 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.593 INFO:teuthology.orchestra.run.smithi070.stdout:4128 punch from 0x42b4010 to 0x42b51f7, (0x11e7 bytes) 2024-09-16T11:44:12.604 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-16T11:44:12.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-16T11:44:12.609 INFO:teuthology.orchestra.run.smithi070.stdout:4130 writesame 0xd1bbd49 thru 0xd1bc6ad data_size 0x5(0x965 bytes) 2024-09-16T11:44:12.609 INFO:teuthology.orchestra.run.smithi070.stdout:4132 write 0x67f90e2 thru 0x6802317 (0x9236 bytes) 2024-09-16T11:44:12.613 INFO:teuthology.orchestra.run.smithi070.stdout:4133 read 0x7236b3e thru 0x723c5d4 (0x5a97 bytes) 2024-09-16T11:44:12.615 INFO:teuthology.orchestra.run.smithi070.stdout:4134 trunc from 0xd1bc6ae to 0xb782afc 2024-09-16T11:44:12.616 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:12.614+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:12.620 INFO:teuthology.orchestra.run.smithi070.stdout:4135 punch from 0x97977f1 to 0x979ff73, (0x8782 bytes) 2024-09-16T11:44:12.620 INFO:teuthology.orchestra.run.smithi070.stdout:4137 writesame 0x9ab6c66 thru 0x9ac569e data_size 0x21(0xea39 bytes) 2024-09-16T11:44:12.625 INFO:teuthology.orchestra.run.smithi070.stdout:4138 write 0x5ef803a thru 0x5f02ada (0xaaa1 bytes) 2024-09-16T11:44:12.638 INFO:teuthology.orchestra.run.smithi070.stdout:4139 write 0xe13b11f thru 0xe14051a (0x53fc bytes) 2024-09-16T11:44:12.651 INFO:teuthology.orchestra.run.smithi070.stdout:4140 trunc from 0xe14051b to 0x6df90b9 2024-09-16T11:44:12.668 INFO:teuthology.orchestra.run.smithi070.stdout:4141 trunc from 0x6df90b9 to 0x11cf736 2024-09-16T11:44:12.691 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:12.691 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:12.696 INFO:teuthology.orchestra.run.smithi070.stdout:4144 write 0xb7e9873 thru 0xb7f8f30 (0xf6be bytes) 2024-09-16T11:44:12.700 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:12.701 INFO:teuthology.orchestra.run.smithi070.stdout:4148 punch from 0x4c61573 to 0x4c682b4, (0x6d41 bytes) 2024-09-16T11:44:12.708 INFO:teuthology.orchestra.run.smithi070.stdout:4150 writesame 0xda13919 thru 0xda18371 data_size 0x1(0x4a59 bytes) 2024-09-16T11:44:12.708 INFO:teuthology.orchestra.run.smithi070.stdout:4151 write 0x1534e91 thru 0x154266f (0xd7df bytes) 2024-09-16T11:44:12.713 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.713 INFO:teuthology.orchestra.run.smithi070.stdout:4153 punch from 0x8719bd1 to 0x8728017, (0xe446 bytes) 2024-09-16T11:44:12.717 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:12.715+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:12.719 INFO:teuthology.orchestra.run.smithi070.stdout:4154 write 0xed48f96 thru 0xed4bfe0 (0x304b bytes) 2024-09-16T11:44:12.723 INFO:teuthology.orchestra.run.smithi070.stdout:4155 punch from 0x765b942 to 0x765e5d8, (0x2c96 bytes) 2024-09-16T11:44:12.723 INFO:teuthology.orchestra.run.smithi070.stdout:4156 read 0x1831fb1 thru 0x1837dc9 (0x5e19 bytes) 2024-09-16T11:44:12.723 INFO:teuthology.orchestra.run.smithi070.stdout:4157 writesame 0x1743bed thru 0x1753535 data_size 0x1(0xf949 bytes) 2024-09-16T11:44:12.741 INFO:teuthology.orchestra.run.smithi070.stdout:4158 write 0x8ba5dc3 thru 0x8ba63af (0x5ed bytes) 2024-09-16T11:44:12.744 INFO:teuthology.orchestra.run.smithi070.stdout:4159 punch from 0xa5a3a59 to 0xa5ac7d5, (0x8d7c bytes) 2024-09-16T11:44:12.744 INFO:teuthology.orchestra.run.smithi070.stdout:4160 punch from 0x9586447 to 0x958daf0, (0x76a9 bytes) 2024-09-16T11:44:12.744 INFO:teuthology.orchestra.run.smithi070.stdout:4162 writesame 0x29bed11 thru 0x29c633d data_size 0x1(0x762d bytes) 2024-09-16T11:44:12.756 INFO:teuthology.orchestra.run.smithi070.stdout:4163 read 0x8fb0065 thru 0x8fba1b8 (0xa154 bytes) 2024-09-16T11:44:12.756 INFO:teuthology.orchestra.run.smithi070.stdout:4164 writesame 0x52ee849 thru 0x52fa60f data_size 0x1(0xbdc7 bytes) 2024-09-16T11:44:12.771 INFO:teuthology.orchestra.run.smithi070.stdout:4165 writesame 0x47ae5e4 thru 0x47b66a0 data_size 0x1(0x80bd bytes) 2024-09-16T11:44:12.801 INFO:teuthology.orchestra.run.smithi070.stdout:4166 trunc from 0xed4bfe1 to 0x19857a3 2024-09-16T11:44:12.802 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:12.802 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:12.812 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:12.818 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:12.816+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:12.826 INFO:teuthology.orchestra.run.smithi070.stdout:4168 write 0x7c3b315 thru 0x7c45afb (0xa7e7 bytes) 2024-09-16T11:44:12.830 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.833 INFO:teuthology.orchestra.run.smithi070.stdout:4171 trunc from 0x7c45afc to 0x9e538e8 2024-09-16T11:44:12.845 INFO:teuthology.orchestra.run.smithi070.stdout:4172 trunc from 0x9e538e8 to 0x3c89e6a 2024-09-16T11:44:12.865 INFO:teuthology.orchestra.run.smithi070.stdout:4173 write 0x8613c9d thru 0x861b2bb (0x761f bytes) 2024-09-16T11:44:12.869 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.870 INFO:teuthology.orchestra.run.smithi070.stdout:4175 read 0x1fb41d8 thru 0x1fc0755 (0xc57e bytes) 2024-09-16T11:44:12.881 INFO:teuthology.orchestra.run.smithi070.stdout:4176 trunc from 0x861b2bc to 0x20bc19 2024-09-16T11:44:12.894 INFO:teuthology.orchestra.run.smithi070.stdout:4177 read 0x33a4f thru 0x3ae96 (0x7448 bytes) 2024-09-16T11:44:12.894 INFO:teuthology.orchestra.run.smithi070.stdout:4178 punch from 0x109f43 to 0x111a78, (0x7b35 bytes) 2024-09-16T11:44:12.895 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.897 INFO:teuthology.orchestra.run.smithi070.stdout:4180 trunc from 0x20bc19 to 0x2005f1f 2024-09-16T11:44:12.908 INFO:teuthology.orchestra.run.smithi070.stdout:4183 writesame 0x36b7763 thru 0x36c5453 data_size 0x1(0xdcf1 bytes) 2024-09-16T11:44:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:12.913 INFO:teuthology.orchestra.run.smithi070.stdout:4184 write 0x3899833 thru 0x38a40c6 (0xa894 bytes) 2024-09-16T11:44:12.917 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.918 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:12.916+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:12.921 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:12.924 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T11:44:12.931 INFO:teuthology.orchestra.run.smithi070.stdout:4189 write 0x6dcdbd6 thru 0x6dd1cf0 (0x411b bytes) 2024-09-16T11:44:12.935 INFO:teuthology.orchestra.run.smithi070.stdout:4190 punch from 0x6bea692 to 0x6bf3783, (0x90f1 bytes) 2024-09-16T11:44:12.935 INFO:teuthology.orchestra.run.smithi070.stdout:4191 read 0x21499d3 thru 0x2157a7f (0xe0ad bytes) 2024-09-16T11:44:12.943 INFO:teuthology.orchestra.run.smithi070.stdout:4193 write 0x908c36e thru 0x9098804 (0xc497 bytes) 2024-09-16T11:44:12.948 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:12.954 INFO:teuthology.orchestra.run.smithi070.stdout:4196 trunc from 0x9098805 to 0x359b3fb 2024-09-16T11:44:12.956 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:44:12.956 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:44:12.956 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.52668 -> 3.52668 (expected baseline 3.13748) 2024-09-16T11:44:12.956 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:44:12.956 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 19 -> 19 pgs (1.68889 -> 1.68889 * mean) 2024-09-16T11:44:12.957 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:12.957 INFO:teuthology.orchestra.run.smithi070.stdout:oload 1000 2024-09-16T11:44:12.957 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 0.05 2024-09-16T11:44:12.957 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:44:12.957 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0033 2024-09-16T11:44:12.957 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0327 2024-09-16T11:44:12.957 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:44:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:4197 read 0x27d3ae thru 0x281c96 (0x48e9 bytes) 2024-09-16T11:44:12.970 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T11:44:12.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-16T11:44:12.980 INFO:teuthology.orchestra.run.smithi070.stdout:4198 write 0xb4d46ee thru 0xb4dac5a (0x656d bytes) 2024-09-16T11:44:12.983 INFO:teuthology.orchestra.run.smithi070.stdout:4199 write 0x3b59f32 thru 0x3b671a3 (0xd272 bytes) 2024-09-16T11:44:12.987 INFO:teuthology.orchestra.run.smithi070.stdout:4200 write 0x1f1d6a8 thru 0x1f25052 (0x79ab bytes) 2024-09-16T11:44:12.996 INFO:teuthology.orchestra.run.smithi070.stdout:4202 trunc from 0xb4dac5b to 0x80f55aa 2024-09-16T11:44:13.008 INFO:teuthology.orchestra.run.smithi070.stdout:4203 trunc from 0x80f55aa to 0xaebdebc 2024-09-16T11:44:13.012 INFO:teuthology.orchestra.run.smithi070.stdout:4204 writesame 0xaad6dd4 thru 0xaad6e2b data_size 0x4(0x58 bytes) 2024-09-16T11:44:13.016 INFO:teuthology.orchestra.run.smithi070.stdout:4206 read 0x8f909ee thru 0x8f9ba5b (0xb06e bytes) 2024-09-16T11:44:13.016 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:13.019 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:13.017+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:13.025 INFO:teuthology.orchestra.run.smithi070.stdout:4208 trunc from 0xaebdebc to 0x3f985f5 2024-09-16T11:44:13.034 INFO:teuthology.orchestra.run.smithi070.stdout:4209 read 0x9e6671 thru 0x9ee170 (0x7b00 bytes) 2024-09-16T11:44:13.034 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:13.049 INFO:teuthology.orchestra.run.smithi070.stdout:4211 writesame 0xcb9f9c3 thru 0xcba4c7a data_size 0x4(0x52b8 bytes) 2024-09-16T11:44:13.050 INFO:teuthology.orchestra.run.smithi070.stdout:4212 punch from 0x2510250 to 0x251361a, (0x33ca bytes) 2024-09-16T11:44:13.059 INFO:teuthology.orchestra.run.smithi070.stdout:4213 trunc from 0xcba4c7b to 0x461490b 2024-09-16T11:44:13.061 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:13.061 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:13.064 INFO:teuthology.orchestra.run.smithi070.stdout:4214 read 0x3d43260 thru 0x3d5321b (0xffbc bytes) 2024-09-16T11:44:13.070 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:13.071 INFO:teuthology.orchestra.run.smithi070.stdout:4215 write 0x4fea751 thru 0x4ff715d (0xca0d bytes) 2024-09-16T11:44:13.075 INFO:teuthology.orchestra.run.smithi070.stdout:4216 write 0x32a7ec5 thru 0x32b7d99 (0xfed5 bytes) 2024-09-16T11:44:13.080 INFO:teuthology.orchestra.run.smithi070.stdout:4217 punch from 0x41dd540 to 0x41e39f0, (0x64b0 bytes) 2024-09-16T11:44:13.080 INFO:teuthology.orchestra.run.smithi070.stdout:4218 read 0x4a13b71 thru 0x4a1c4bd (0x894d bytes) 2024-09-16T11:44:13.091 INFO:teuthology.orchestra.run.smithi070.stdout:4222 writesame 0x9cd1e9b thru 0x9cd5077 data_size 0x25(0x31dd bytes) 2024-09-16T11:44:13.092 INFO:teuthology.orchestra.run.smithi070.stdout:4225 read 0x4678edf thru 0x467d3d8 (0x44fa bytes) 2024-09-16T11:44:13.092 INFO:teuthology.orchestra.run.smithi070.stdout:4226 read 0xcb611c thru 0xcc3fb4 (0xde99 bytes) 2024-09-16T11:44:13.092 INFO:teuthology.orchestra.run.smithi070.stdout:4228 writesame 0x86d8312 thru 0x86dce62 data_size 0x1(0x4b51 bytes) 2024-09-16T11:44:13.103 INFO:teuthology.orchestra.run.smithi070.stdout:4229 read 0x8792517 thru 0x87a1952 (0xf43c bytes) 2024-09-16T11:44:13.103 INFO:teuthology.orchestra.run.smithi070.stdout:4230 writesame 0x4fca8ee thru 0x4fcd6db data_size 0x1(0x2dee bytes) 2024-09-16T11:44:13.107 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:13.107 INFO:teuthology.orchestra.run.smithi070.stdout:4232 write 0x87a65ee thru 0x87b2910 (0xc323 bytes) 2024-09-16T11:44:13.120 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:13.118+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:13.121 INFO:teuthology.orchestra.run.smithi070.stdout:4234 writesame 0xd6f62ff thru 0xd704341 data_size 0x3(0xe043 bytes) 2024-09-16T11:44:13.121 INFO:teuthology.orchestra.run.smithi070.stdout:4236 trunc from 0xd704342 to 0xd9d888c 2024-09-16T11:44:13.126 INFO:teuthology.orchestra.run.smithi070.stdout:4238 read 0x2387622 thru 0x2389978 (0x2357 bytes) 2024-09-16T11:44:13.126 INFO:teuthology.orchestra.run.smithi070.stdout:4239 read 0x62559ac thru 0x6256307 (0x95c bytes) 2024-09-16T11:44:13.126 INFO:teuthology.orchestra.run.smithi070.stdout:4240 read 0xa1035e thru 0xa182ea (0x7f8d bytes) 2024-09-16T11:44:13.126 INFO:teuthology.orchestra.run.smithi070.stdout:4241 write 0x649c4de thru 0x64a2040 (0x5b63 bytes) 2024-09-16T11:44:13.131 INFO:teuthology.orchestra.run.smithi070.stdout:4242 read 0x15449b4 thru 0x154c7be (0x7e0b bytes) 2024-09-16T11:44:13.131 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:13.131 INFO:teuthology.orchestra.run.smithi070.stdout:4245 read 0xbb1d37d thru 0xbb2a636 (0xd2ba bytes) 2024-09-16T11:44:13.131 INFO:teuthology.orchestra.run.smithi070.stdout:4247 read 0xc77c90d thru 0xc7871ba (0xa8ae bytes) 2024-09-16T11:44:13.131 INFO:teuthology.orchestra.run.smithi070.stdout:4249 write 0xa767e25 thru 0xa77630f (0xe4eb bytes) 2024-09-16T11:44:13.135 INFO:teuthology.orchestra.run.smithi070.stdout:4250 write 0x5c1100a thru 0x5c12cbc (0x1cb3 bytes) 2024-09-16T11:44:13.153 INFO:teuthology.orchestra.run.smithi070.stdout:4251 trunc from 0xd9d888c to 0x422d6e6 2024-09-16T11:44:13.166 INFO:teuthology.orchestra.run.smithi070.stdout:4253 read 0x2861c36 thru 0x2869ddd (0x81a8 bytes) 2024-09-16T11:44:13.183 INFO:teuthology.orchestra.run.smithi070.stdout:4254 write 0xb499409 thru 0xb49b8b8 (0x24b0 bytes) 2024-09-16T11:44:13.185 INFO:teuthology.orchestra.run.smithi070.stdout:4255 writesame 0x38fff75 thru 0x390371e data_size 0x4b(0x37aa bytes) 2024-09-16T11:44:13.187 INFO:teuthology.orchestra.run.smithi070.stdout:4257 clone 26 order 22 su 524288 sc 7 2024-09-16T11:44:13.191 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:13.191 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:13.200 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:13.241 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:13.239+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:13.299 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:44:13.299 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:44:13.299 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.52668 -> 3.52668 (expected baseline 3.13748) 2024-09-16T11:44:13.299 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:44:13.300 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 19 -> 19 pgs (1.68889 -> 1.68889 * mean) 2024-09-16T11:44:13.300 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:13.300 INFO:teuthology.orchestra.run.smithi070.stdout:oload 1000 2024-09-16T11:44:13.300 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 1 2024-09-16T11:44:13.300 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:44:13.300 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0033 2024-09-16T11:44:13.301 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0327 2024-09-16T11:44:13.301 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:44:13.312 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T11:44:13.312 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-16T11:44:13.341 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:13.341 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:13.341 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:13.339+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:13.350 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:13.423 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:13.420+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:13.490 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:13.490 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:13.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:13.524 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:13.522+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:13.582 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:44:13.582 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:44:13.583 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:44:13.583 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.52668 -> 3.52668 (expected baseline 3.13748) 2024-09-16T11:44:13.583 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:44:13.583 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 19 -> 19 pgs (1.68889 -> 1.68889 * mean) 2024-09-16T11:44:13.583 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:13.583 INFO:teuthology.orchestra.run.smithi070.stdout:oload 110 2024-09-16T11:44:13.583 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 0.05 2024-09-16T11:44:13.583 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:44:13.583 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0033 2024-09-16T11:44:13.583 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0036 2024-09-16T11:44:13.584 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:44:13.597 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:13.597 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:13.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:13.626 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:13.624+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:13.699 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:13.699 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:13.715 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:13.715 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:13.726 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:13.725+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:13.729 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:13.730 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:13.744 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:13.745 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:13.747 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:13.747 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:13.747 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:13.747 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:13.747 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:13.756 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:13.760 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:13.760 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:13.776 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:13.776 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:13.793 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:13.793 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:13.808 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:13.856 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:13.856 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:13.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:13.856 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19364 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1171456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.340457443999998, 2024-09-16T11:44:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049001805000000002, 2024-09-16T11:44:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19364 2024-09-16T11:44:13.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513280+0000", 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7090000000000001e-06 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513286+0000", 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5840000000000002e-06 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513290+0000", 2024-09-16T11:44:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3680000000000004e-06 2024-09-16T11:44:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562121+0000", 2024-09-16T11:44:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048831324000000002 2024-09-16T11:44:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562195+0000", 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3415000000000001e-05 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562280+0000", 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4975999999999994e-05 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19365 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1490944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.861 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.340437704999999, 2024-09-16T11:44:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049335349000000001, 2024-09-16T11:44:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19365 2024-09-16T11:44:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513300+0000", 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3020000000000002e-06 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513306+0000", 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8780000000000003e-06 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513310+0000", 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1309999999999999e-06 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562322+0000", 2024-09-16T11:44:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049011635999999997 2024-09-16T11:44:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562544+0000", 2024-09-16T11:44:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022168000000000001 2024-09-16T11:44:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562633+0000", 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9350000000000003e-05 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19366 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1916928~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.340420773000002, 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049865073000000003, 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.866 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19366 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515102+0000", 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017877419999999999 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515106+0000", 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4450000000000001e-06 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515113+0000", 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7680000000000008e-06 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515117+0000", 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562756+0000", 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047638579 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563093+0000", 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000337516 2024-09-16T11:44:13.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563180+0000", 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6420000000000003e-05 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19371 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3194880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.338523565999999, 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048948356999999998, 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.870 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19371 2024-09-16T11:44:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.872 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515214+0000", 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5969999999999999e-06 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515219+0000", 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4559999999999999e-06 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515223+0000", 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563991+0000", 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048768911999999998 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564063+0000", 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2010999999999999e-05 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564160+0000", 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6617e-05 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.874 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19372 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3620864~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.338509348999999, 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049130502, 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19372 2024-09-16T11:44:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.876 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:44:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515228+0000", 2024-09-16T11:44:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T11:44:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515233+0000", 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515237+0000", 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.27e-06 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564199+0000", 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048961787999999999 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564275+0000", 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6001000000000006e-05 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564356+0000", 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1941999999999996e-05 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19373 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3940352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:13.879 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.338494949000001, 2024-09-16T11:44:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049289588000000002, 2024-09-16T11:44:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19373 2024-09-16T11:44:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.881 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515241+0000", 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515242+0000", 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.528e-06 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515248+0000", 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515251+0000", 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2480000000000001e-06 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564388+0000", 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049137313000000002 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564455+0000", 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7049000000000001e-05 2024-09-16T11:44:13.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564530+0000", 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4626e-05 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19374 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 4046848~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.338480546, 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049457133, 2024-09-16T11:44:13.884 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19374 2024-09-16T11:44:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515257+0000", 2024-09-16T11:44:13.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5740000000000001e-06 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515262+0000", 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2399999999999998e-06 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515265+0000", 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3909999999999998e-06 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564555+0000", 2024-09-16T11:44:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049290002999999999 2024-09-16T11:44:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564628+0000", 2024-09-16T11:44:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2559999999999996e-05 2024-09-16T11:44:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564712+0000", 2024-09-16T11:44:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4040000000000002e-05 2024-09-16T11:44:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.888 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19375 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3301376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.338466403000002, 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049606631999999998, 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19375 2024-09-16T11:44:13.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:13.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:44:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515271+0000", 2024-09-16T11:44:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5230000000000001e-06 2024-09-16T11:44:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515276+0000", 2024-09-16T11:44:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4650000000000002e-06 2024-09-16T11:44:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515279+0000", 2024-09-16T11:44:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0630000000000002e-06 2024-09-16T11:44:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564738+0000", 2024-09-16T11:44:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049459001000000002 2024-09-16T11:44:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564780+0000", 2024-09-16T11:44:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1693999999999998e-05 2024-09-16T11:44:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564876+0000", 2024-09-16T11:44:13.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5571999999999999e-05 2024-09-16T11:44:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19376 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3727360~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.338452491000002, 2024-09-16T11:44:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049720590000000002, 2024-09-16T11:44:13.894 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19376 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.895 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517103+0000", 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001820276 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517108+0000", 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5410000000000002e-06 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517120+0000", 2024-09-16T11:44:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2092e-05 2024-09-16T11:44:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517125+0000", 2024-09-16T11:44:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6829999999999996e-06 2024-09-16T11:44:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564870+0000", 2024-09-16T11:44:13.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047745040000000002 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564921+0000", 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0538999999999998e-05 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565003+0000", 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2418999999999998e-05 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19492 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 434176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.336523634999999, 2024-09-16T11:44:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048926948999999997, 2024-09-16T11:44:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19492 2024-09-16T11:44:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.899 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:13.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3099999999999999e-07 2024-09-16T11:44:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517213+0000", 2024-09-16T11:44:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6599999999999994e-07 2024-09-16T11:44:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517216+0000", 2024-09-16T11:44:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.706e-06 2024-09-16T11:44:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517218+0000", 2024-09-16T11:44:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.035e-06 2024-09-16T11:44:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565959+0000", 2024-09-16T11:44:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048741380000000001 2024-09-16T11:44:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566036+0000", 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7362999999999998e-05 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566139+0000", 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102268 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19493 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 860160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:13.903 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.336501964, 2024-09-16T11:44:13.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049086604999999998, 2024-09-16T11:44:13.904 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.904 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.904 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.904 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.904 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.904 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19493 2024-09-16T11:44:13.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.904 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:13.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:13.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517234+0000", 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.37e-07 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517236+0000", 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7150000000000001e-06 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517243+0000", 2024-09-16T11:44:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9489999999999999e-06 2024-09-16T11:44:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517246+0000", 2024-09-16T11:44:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.636e-06 2024-09-16T11:44:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566163+0000", 2024-09-16T11:44:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048917284999999998 2024-09-16T11:44:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566237+0000", 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3196999999999997e-05 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566320+0000", 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3286000000000002e-05 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.908 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19494 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1179648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.336481348, 2024-09-16T11:44:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049259562999999999, 2024-09-16T11:44:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19494 2024-09-16T11:44:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:13.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517256+0000", 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.888e-06 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517265+0000", 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8629999999999997e-06 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517269+0000", 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2039999999999999e-06 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566352+0000", 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049083107000000001 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566429+0000", 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7025000000000002e-05 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566514+0000", 2024-09-16T11:44:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4121000000000003e-05 2024-09-16T11:44:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19495 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1286144~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.336461924999998, 2024-09-16T11:44:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049492880000000003, 2024-09-16T11:44:13.913 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19495 2024-09-16T11:44:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517274+0000", 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517276+0000", 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8950000000000001e-06 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517281+0000", 2024-09-16T11:44:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3879999999999999e-06 2024-09-16T11:44:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517285+0000", 2024-09-16T11:44:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5930000000000002e-06 2024-09-16T11:44:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566579+0000", 2024-09-16T11:44:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049294175000000003 2024-09-16T11:44:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566670+0000", 2024-09-16T11:44:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1096999999999999e-05 2024-09-16T11:44:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566766+0000", 2024-09-16T11:44:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6355999999999997e-05 2024-09-16T11:44:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.917 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19496 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1605632~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.336446840000001, 2024-09-16T11:44:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049632981999999999, 2024-09-16T11:44:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.918 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19496 2024-09-16T11:44:13.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.919 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:13.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517289+0000", 2024-09-16T11:44:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517290+0000", 2024-09-16T11:44:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.581e-06 2024-09-16T11:44:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517300+0000", 2024-09-16T11:44:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517304+0000", 2024-09-16T11:44:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:44:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566789+0000", 2024-09-16T11:44:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049485102000000003 2024-09-16T11:44:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566843+0000", 2024-09-16T11:44:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3937000000000002e-05 2024-09-16T11:44:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566921+0000", 2024-09-16T11:44:13.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8397000000000002e-05 2024-09-16T11:44:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19497 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 114688~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.336427144999998, 2024-09-16T11:44:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049761367000000001, 2024-09-16T11:44:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.923 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19497 2024-09-16T11:44:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519116+0000", 2024-09-16T11:44:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018079439999999999 2024-09-16T11:44:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519120+0000", 2024-09-16T11:44:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2760000000000002e-06 2024-09-16T11:44:13.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519134+0000", 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3468999999999999e-05 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519140+0000", 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.516e-06 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566943+0000", 2024-09-16T11:44:13.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047802561 2024-09-16T11:44:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566982+0000", 2024-09-16T11:44:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9269e-05 2024-09-16T11:44:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567070+0000", 2024-09-16T11:44:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7331999999999997e-05 2024-09-16T11:44:13.927 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19504 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 966656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.334498658000001, 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049045863000000002, 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19504 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519239+0000", 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.962e-06 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519244+0000", 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5480000000000001e-06 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519248+0000", 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.641e-06 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568150+0000", 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048901864000000003 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568208+0000", 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8399000000000001e-05 2024-09-16T11:44:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568283+0000", 2024-09-16T11:44:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4105999999999998e-05 2024-09-16T11:44:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19505 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1392640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.334483353, 2024-09-16T11:44:13.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049202304000000002, 2024-09-16T11:44:13.933 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.933 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.933 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.933 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.933 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.933 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19505 2024-09-16T11:44:13.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.933 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:44:13.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519254+0000", 2024-09-16T11:44:13.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.761e-06 2024-09-16T11:44:13.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519257+0000", 2024-09-16T11:44:13.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.343e-06 2024-09-16T11:44:13.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519259+0000", 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0559999999999999e-06 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568307+0000", 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049047445000000002 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568382+0000", 2024-09-16T11:44:13.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5574999999999994e-05 2024-09-16T11:44:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568454+0000", 2024-09-16T11:44:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1799999999999997e-05 2024-09-16T11:44:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.937 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19506 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1712128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.334473315, 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049341532, 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19506 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521102+0000", 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018404700000000001 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521106+0000", 2024-09-16T11:44:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8820000000000003e-06 2024-09-16T11:44:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521119+0000", 2024-09-16T11:44:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2546e-05 2024-09-16T11:44:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521125+0000", 2024-09-16T11:44:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0249999999999999e-06 2024-09-16T11:44:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568484+0000", 2024-09-16T11:44:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047359120999999997 2024-09-16T11:44:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568542+0000", 2024-09-16T11:44:13.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8360999999999999e-05 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568604+0000", 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1126999999999999e-05 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19515 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1499136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.33248103, 2024-09-16T11:44:13.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048949431000000002, 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19515 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523130+0000", 2024-09-16T11:44:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00187604 2024-09-16T11:44:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523134+0000", 2024-09-16T11:44:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:44:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523144+0000", 2024-09-16T11:44:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2010000000000005e-06 2024-09-16T11:44:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523149+0000", 2024-09-16T11:44:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0039999999999999e-06 2024-09-16T11:44:13.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570113+0000", 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.046964517999999997 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570157+0000", 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4056e-05 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570204+0000", 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6591999999999997e-05 2024-09-16T11:44:13.946 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:44111 4.8 4:17f3f47e:::rbd_data.11c1e170f5f0.000000000000002f:head [delete] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.8705007490000001, 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93903426400000001, 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:13.947 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 44111 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7199999999999998e-07 2024-09-16T11:44:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983239+0000", 2024-09-16T11:44:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9879999999999998e-06 2024-09-16T11:44:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983248+0000", 2024-09-16T11:44:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.371e-06 2024-09-16T11:44:13.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983253+0000", 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1150000000000002e-06 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983303+0000", 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0380000000000002e-05 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983311+0000", 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7519999999999996e-06 2024-09-16T11:44:13.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920618+0000", 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93730672000000004 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920730+0000", 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011265900000000001 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920842+0000", 2024-09-16T11:44:13.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111638 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.921503+0000", 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066074299999999999 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922208+0000", 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00070492899999999997 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:13.952 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922233+0000", 2024-09-16T11:44:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5922999999999999e-05 2024-09-16T11:44:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922269+0000", 2024-09-16T11:44:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5373999999999997e-05 2024-09-16T11:44:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:13.953 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:13.954 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:13.954 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:13.854+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:13.955 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:13.985 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone25 from 0xb49b8b9 (overlap 0x20bc19) to 0xd180a 2024-09-16T11:44:13.986 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:13.983+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:14.034 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-16T11:44:14.057 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:14.055+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:14.108 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:14.109 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:14.109 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:14.109 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:14.118 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:14.158 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:14.156+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:14.217 INFO:teuthology.orchestra.run.smithi070.stdout:4258 punch from 0x1e7cc89 to 0x1e8238b, (0x5702 bytes) 2024-09-16T11:44:14.221 INFO:teuthology.orchestra.run.smithi070.stdout:4261 read 0x4ce5104 thru 0x4ce53d9 (0x2d6 bytes) 2024-09-16T11:44:14.229 INFO:teuthology.orchestra.run.smithi070.stdout:4263 write 0xd253ea6 thru 0xd261449 (0xd5a4 bytes) 2024-09-16T11:44:14.232 INFO:teuthology.orchestra.run.smithi070.stdout:4264 write 0xc8a29a8 thru 0xc8a9145 (0x679e bytes) 2024-09-16T11:44:14.241 INFO:teuthology.orchestra.run.smithi070.stdout:4265 trunc from 0xd26144a to 0x87692e7 2024-09-16T11:44:14.258 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:14.256+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:14.261 INFO:teuthology.orchestra.run.smithi070.stdout:4266 write 0x79af2b3 thru 0x79af78c (0x4da bytes) 2024-09-16T11:44:14.263 INFO:teuthology.orchestra.run.smithi070.stdout:4267 read 0x27993d3 thru 0x27a296a (0x9598 bytes) 2024-09-16T11:44:14.263 INFO:teuthology.orchestra.run.smithi070.stdout:4269 write 0x2acc1b1 thru 0x2adabbf (0xea0f bytes) 2024-09-16T11:44:14.265 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:14.266 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:14.268 INFO:teuthology.orchestra.run.smithi070.stdout:4270 read 0x5d21eab thru 0x5d2d9f5 (0xbb4b bytes) 2024-09-16T11:44:14.269 INFO:teuthology.orchestra.run.smithi070.stdout:4271 write 0x80f165f thru 0x80f2d5f (0x1701 bytes) 2024-09-16T11:44:14.270 INFO:teuthology.orchestra.run.smithi070.stdout:4273 punch from 0x2590171 to 0x259ee55, (0xece4 bytes) 2024-09-16T11:44:14.275 INFO:teuthology.orchestra.run.smithi070.stdout:4274 trunc from 0x87692e7 to 0x47bb7e6 2024-09-16T11:44:14.276 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:14.294 INFO:teuthology.orchestra.run.smithi070.stdout:4276 read 0x362c213 thru 0x363b1cb (0xefb9 bytes) 2024-09-16T11:44:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:4277 punch from 0x35be011 to 0x35c10db, (0x30ca bytes) 2024-09-16T11:44:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:4278 punch from 0x3706b3f to 0x3712740, (0xbc01 bytes) 2024-09-16T11:44:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:4279 writesame 0x1eb95e thru 0x1f3f11 data_size 0x56(0x85b4 bytes) 2024-09-16T11:44:14.300 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:14.300 INFO:teuthology.orchestra.run.smithi070.stdout:4284 read 0x3d04fea thru 0x3d07a4e (0x2a65 bytes) 2024-09-16T11:44:14.301 INFO:teuthology.orchestra.run.smithi070.stdout:4287 punch from 0x20ead78 to 0x20eb278, (0x500 bytes) 2024-09-16T11:44:14.301 INFO:teuthology.orchestra.run.smithi070.stdout:4288 writesame 0x419cc89 thru 0x41abeae data_size 0x5(0xf226 bytes) 2024-09-16T11:44:14.307 INFO:teuthology.orchestra.run.smithi070.stdout:4289 read 0x2e81395 thru 0x2e87cbb (0x6927 bytes) 2024-09-16T11:44:14.307 INFO:teuthology.orchestra.run.smithi070.stdout:4292 punch from 0x20bc72c to 0x20c2747, (0x601b bytes) 2024-09-16T11:44:14.307 INFO:teuthology.orchestra.run.smithi070.stdout:4293 read 0x3db0d79 thru 0x3db989d (0x8b25 bytes) 2024-09-16T11:44:14.316 INFO:teuthology.orchestra.run.smithi070.stdout:4294 write 0x7b18692 thru 0x7b214da (0x8e49 bytes) 2024-09-16T11:44:14.319 INFO:teuthology.orchestra.run.smithi070.stdout:4295 writesame 0x2b5eb86 thru 0x2b6603a data_size 0x17(0x74b5 bytes) 2024-09-16T11:44:14.330 INFO:teuthology.orchestra.run.smithi070.stdout:4296 trunc from 0x7b214db to 0x1c0aebc 2024-09-16T11:44:14.350 INFO:teuthology.orchestra.run.smithi070.stdout:4297 read 0xe3b224 thru 0xe410e7 (0x5ec4 bytes) 2024-09-16T11:44:14.350 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:14.350 INFO:teuthology.orchestra.run.smithi070.stdout:4301 trunc from 0x1c0aebc to 0x245e746 2024-09-16T11:44:14.359 INFO:teuthology.orchestra.run.smithi070.stdout:4302 trunc from 0x245e746 to 0x4f863b8 2024-09-16T11:44:14.359 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:14.357+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:4303 trunc from 0x4f863b8 to 0x440a943 2024-09-16T11:44:14.377 INFO:teuthology.orchestra.run.smithi070.stdout:4304 write 0x529b4ae thru 0x52a1216 (0x5d69 bytes) 2024-09-16T11:44:14.380 INFO:teuthology.orchestra.run.smithi070.stdout:4306 write 0x2a3096d thru 0x2a3b94e (0xafe2 bytes) 2024-09-16T11:44:14.395 INFO:teuthology.orchestra.run.smithi070.stdout:4308 write 0xc6648bb thru 0xc670477 (0xbbbd bytes) 2024-09-16T11:44:14.398 INFO:teuthology.orchestra.run.smithi070.stdout:4309 read 0xaf21fc0 thru 0xaf2cb39 (0xab7a bytes) 2024-09-16T11:44:14.399 INFO:teuthology.orchestra.run.smithi070.stdout:4310 read 0x2c5992b thru 0x2c59a17 (0xed bytes) 2024-09-16T11:44:14.399 INFO:teuthology.orchestra.run.smithi070.stdout:4312 punch from 0x6bdc9c0 to 0x6be1d9c, (0x53dc bytes) 2024-09-16T11:44:14.399 INFO:teuthology.orchestra.run.smithi070.stdout:4313 read 0x5a655d7 thru 0x5a74cd6 (0xf700 bytes) 2024-09-16T11:44:14.399 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:14.399 INFO:teuthology.orchestra.run.smithi070.stdout:4315 writesame 0x9dd1208 thru 0x9dd3802 data_size 0x15(0x25fb bytes) 2024-09-16T11:44:14.404 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:14.404 INFO:teuthology.orchestra.run.smithi070.stdout:4318 clone 27 order 21 su 1048576 sc 9 2024-09-16T11:44:14.405 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:14.405 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:14.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16853 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3375104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.055487927000001, 2024-09-16T11:44:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027791907000000001, 2024-09-16T11:44:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.406 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16853 2024-09-16T11:44:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347568+0000", 2024-09-16T11:44:14.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:44:14.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347570+0000", 2024-09-16T11:44:14.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:44:14.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347588+0000", 2024-09-16T11:44:14.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7914000000000002e-05 2024-09-16T11:44:14.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347592+0000", 2024-09-16T11:44:14.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3150000000000002e-06 2024-09-16T11:44:14.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375199+0000", 2024-09-16T11:44:14.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027606065999999999 2024-09-16T11:44:14.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375276+0000", 2024-09-16T11:44:14.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7527000000000005e-05 2024-09-16T11:44:14.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375359+0000", 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3238000000000005e-05 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16854 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.055458229999999, 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027945148999999999, 2024-09-16T11:44:14.411 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.412 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.412 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.412 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.412 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.412 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16854 2024-09-16T11:44:14.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.412 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:14.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:14.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347598+0000", 2024-09-16T11:44:14.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:44:14.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347600+0000", 2024-09-16T11:44:14.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:44:14.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347606+0000", 2024-09-16T11:44:14.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0580000000000003e-06 2024-09-16T11:44:14.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347609+0000", 2024-09-16T11:44:14.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:44:14.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375386+0000", 2024-09-16T11:44:14.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027776809 2024-09-16T11:44:14.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375461+0000", 2024-09-16T11:44:14.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4868000000000002e-05 2024-09-16T11:44:14.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375542+0000", 2024-09-16T11:44:14.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1253000000000003e-05 2024-09-16T11:44:14.418 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.418 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.418 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.418 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16855 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 4120576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.418 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:14.419 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.055441891000001, 2024-09-16T11:44:14.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028111094999999999, 2024-09-16T11:44:14.419 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.419 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.419 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.419 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.419 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.419 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16855 2024-09-16T11:44:14.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.420 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:14.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:14.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347614+0000", 2024-09-16T11:44:14.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:44:14.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347616+0000", 2024-09-16T11:44:14.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:44:14.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347623+0000", 2024-09-16T11:44:14.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0099999999999998e-06 2024-09-16T11:44:14.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347626+0000", 2024-09-16T11:44:14.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2140000000000001e-06 2024-09-16T11:44:14.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375569+0000", 2024-09-16T11:44:14.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027942649999999999 2024-09-16T11:44:14.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375643+0000", 2024-09-16T11:44:14.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3949999999999995e-05 2024-09-16T11:44:14.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375725+0000", 2024-09-16T11:44:14.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1992999999999999e-05 2024-09-16T11:44:14.425 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.425 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.425 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.426 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16856 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3801088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.426 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:14.426 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.055419820000001, 2024-09-16T11:44:14.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028268272000000001, 2024-09-16T11:44:14.426 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.426 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.426 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16856 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:14.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347636+0000", 2024-09-16T11:44:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:44:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347638+0000", 2024-09-16T11:44:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.86e-06 2024-09-16T11:44:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347644+0000", 2024-09-16T11:44:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1739999999999997e-06 2024-09-16T11:44:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347656+0000", 2024-09-16T11:44:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2198000000000001e-05 2024-09-16T11:44:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375751+0000", 2024-09-16T11:44:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028094507000000001 2024-09-16T11:44:14.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375824+0000", 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3178999999999998e-05 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375904+0000", 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9985000000000001e-05 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16857 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.055393318, 2024-09-16T11:44:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028275403000000001, 2024-09-16T11:44:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16857 2024-09-16T11:44:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:14.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:14.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:14.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.46e-07 2024-09-16T11:44:14.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347666+0000", 2024-09-16T11:44:14.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8739999999999997e-06 2024-09-16T11:44:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347672+0000", 2024-09-16T11:44:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1059999999999996e-06 2024-09-16T11:44:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347676+0000", 2024-09-16T11:44:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:44:14.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375930+0000", 2024-09-16T11:44:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028254042 2024-09-16T11:44:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375937+0000", 2024-09-16T11:44:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5519999999999999e-06 2024-09-16T11:44:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16858 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.055375673, 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028284592000000001, 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16858 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:44:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347683+0000", 2024-09-16T11:44:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6809999999999998e-06 2024-09-16T11:44:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347689+0000", 2024-09-16T11:44:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347695+0000", 2024-09-16T11:44:14.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0020000000000003e-06 2024-09-16T11:44:14.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375958+0000", 2024-09-16T11:44:14.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028263870999999999 2024-09-16T11:44:14.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375964+0000", 2024-09-16T11:44:14.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8209999999999998e-06 2024-09-16T11:44:14.440 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.440 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.440 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.440 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16992 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.440 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:14.440 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.053800984999999, 2024-09-16T11:44:14.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027704776, 2024-09-16T11:44:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16992 2024-09-16T11:44:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:14.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349255+0000", 2024-09-16T11:44:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:44:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349260+0000", 2024-09-16T11:44:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5589999999999996e-06 2024-09-16T11:44:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349267+0000", 2024-09-16T11:44:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4019999999999999e-06 2024-09-16T11:44:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349270+0000", 2024-09-16T11:44:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6780000000000002e-06 2024-09-16T11:44:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376953+0000", 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682933 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376959+0000", 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16993 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.05377966, 2024-09-16T11:44:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027707908999999999, 2024-09-16T11:44:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16993 2024-09-16T11:44:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:14.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349278+0000", 2024-09-16T11:44:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.629e-06 2024-09-16T11:44:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349291+0000", 2024-09-16T11:44:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614e-05 2024-09-16T11:44:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349295+0000", 2024-09-16T11:44:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1189999999999999e-06 2024-09-16T11:44:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376978+0000", 2024-09-16T11:44:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682445 2024-09-16T11:44:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376984+0000", 2024-09-16T11:44:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7590000000000003e-06 2024-09-16T11:44:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.448 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16994 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.053755648999999, 2024-09-16T11:44:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027708753999999999, 2024-09-16T11:44:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.449 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16994 2024-09-16T11:44:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349302+0000", 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.722e-06 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349308+0000", 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:44:14.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349312+0000", 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5310000000000002e-06 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377003+0000", 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027691308000000001 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377008+0000", 2024-09-16T11:44:14.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4550000000000003e-06 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16999 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.053612771000001, 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027705270000000001, 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.453 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16999 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.46e-07 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349445+0000", 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.637e-06 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349451+0000", 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9959999999999999e-06 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349454+0000", 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.647e-06 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377142+0000", 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027687942 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377148+0000", 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7019999999999998e-06 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.456 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17000 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.053597264, 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.02771881, 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17000 2024-09-16T11:44:14.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0600000000000001e-07 2024-09-16T11:44:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349460+0000", 2024-09-16T11:44:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.736e-06 2024-09-16T11:44:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349466+0000", 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2419999999999997e-06 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349470+0000", 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5480000000000002e-06 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377171+0000", 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027701282000000001 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.460 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377177+0000", 2024-09-16T11:44:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5960000000000003e-06 2024-09-16T11:44:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17001 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.053581514000001, 2024-09-16T11:44:14.461 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027739243, 2024-09-16T11:44:14.462 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.462 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:14.462 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.462 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.462 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.462 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17001 2024-09-16T11:44:14.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.462 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:14.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:14.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349476+0000", 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349482+0000", 2024-09-16T11:44:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2389999999999999e-06 2024-09-16T11:44:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349486+0000", 2024-09-16T11:44:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5980000000000001e-06 2024-09-16T11:44:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377206+0000", 2024-09-16T11:44:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027720037999999999 2024-09-16T11:44:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377213+0000", 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3440000000000003e-06 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17433 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 5898240~4888] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.017617860000001, 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027709398, 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.466 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17433 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:14.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385438+0000", 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385439+0000", 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5120000000000001e-06 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385445+0000", 2024-09-16T11:44:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4e-06 2024-09-16T11:44:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385448+0000", 2024-09-16T11:44:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1669999999999999e-06 2024-09-16T11:44:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412916+0000", 2024-09-16T11:44:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027468137 2024-09-16T11:44:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412988+0000", 2024-09-16T11:44:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1606000000000005e-05 2024-09-16T11:44:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413147+0000", 2024-09-16T11:44:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015924299999999999 2024-09-16T11:44:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17434 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6004736~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.017603269999999, 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027777461, 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17434 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:14.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385454+0000", 2024-09-16T11:44:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6729999999999999e-06 2024-09-16T11:44:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385459+0000", 2024-09-16T11:44:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4079999999999997e-06 2024-09-16T11:44:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385463+0000", 2024-09-16T11:44:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.269e-06 2024-09-16T11:44:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413157+0000", 2024-09-16T11:44:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027694611000000001 2024-09-16T11:44:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413193+0000", 2024-09-16T11:44:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5673999999999998e-05 2024-09-16T11:44:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413230+0000", 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6454e-05 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17435 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6324224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.017589117, 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929059999999999, 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.475 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.476 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.476 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.476 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.476 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17435 2024-09-16T11:44:14.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.476 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:14.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:14.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385467+0000", 2024-09-16T11:44:14.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:14.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385468+0000", 2024-09-16T11:44:14.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6300000000000001e-06 2024-09-16T11:44:14.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385474+0000", 2024-09-16T11:44:14.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1299999999999998e-06 2024-09-16T11:44:14.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385478+0000", 2024-09-16T11:44:14.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2880000000000001e-06 2024-09-16T11:44:14.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413257+0000", 2024-09-16T11:44:14.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027779608000000001 2024-09-16T11:44:14.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413331+0000", 2024-09-16T11:44:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3639000000000004e-05 2024-09-16T11:44:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413395+0000", 2024-09-16T11:44:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4424999999999994e-05 2024-09-16T11:44:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.483 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17436 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7176192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.483 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:14.483 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.017573455000001, 2024-09-16T11:44:14.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028016718999999999, 2024-09-16T11:44:14.483 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.483 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.483 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17436 2024-09-16T11:44:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385484+0000", 2024-09-16T11:44:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5179999999999999e-06 2024-09-16T11:44:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385489+0000", 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7049999999999996e-06 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385493+0000", 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413422+0000", 2024-09-16T11:44:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929611999999999 2024-09-16T11:44:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413462+0000", 2024-09-16T11:44:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9585000000000001e-05 2024-09-16T11:44:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413499+0000", 2024-09-16T11:44:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6644999999999999e-05 2024-09-16T11:44:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.487 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17437 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6750208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.017559039999998, 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028167168999999999, 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17437 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.488 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385497+0000", 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385498+0000", 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.592e-06 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385504+0000", 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4090000000000002e-06 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385513+0000", 2024-09-16T11:44:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4460000000000007e-06 2024-09-16T11:44:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413526+0000", 2024-09-16T11:44:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028012866000000001 2024-09-16T11:44:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413601+0000", 2024-09-16T11:44:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5388000000000004e-05 2024-09-16T11:44:14.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413663+0000", 2024-09-16T11:44:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.211e-05 2024-09-16T11:44:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17438 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7495680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.017536281000002, 2024-09-16T11:44:14.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028224439, 2024-09-16T11:44:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17438 2024-09-16T11:44:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4900000000000001e-07 2024-09-16T11:44:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385521+0000", 2024-09-16T11:44:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:44:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385526+0000", 2024-09-16T11:44:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2680000000000002e-06 2024-09-16T11:44:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385529+0000", 2024-09-16T11:44:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2040000000000002e-06 2024-09-16T11:44:14.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413674+0000", 2024-09-16T11:44:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028144226000000001 2024-09-16T11:44:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413710+0000", 2024-09-16T11:44:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6665000000000002e-05 2024-09-16T11:44:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413743+0000", 2024-09-16T11:44:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3155999999999999e-05 2024-09-16T11:44:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17439 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7921664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.017475127000001, 2024-09-16T11:44:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028326131000000001, 2024-09-16T11:44:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.497 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17439 2024-09-16T11:44:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385581+0000", 2024-09-16T11:44:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9299999999999998e-07 2024-09-16T11:44:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385583+0000", 2024-09-16T11:44:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6220000000000001e-06 2024-09-16T11:44:14.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385592+0000", 2024-09-16T11:44:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5220000000000007e-06 2024-09-16T11:44:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385595+0000", 2024-09-16T11:44:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8669999999999998e-06 2024-09-16T11:44:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413771+0000", 2024-09-16T11:44:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028175875999999999 2024-09-16T11:44:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413845+0000", 2024-09-16T11:44:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4382999999999996e-05 2024-09-16T11:44:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413906+0000", 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1367999999999999e-05 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17440 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.017457919000002, 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028391117, 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.502 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17440 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385598+0000", 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385603+0000", 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385607+0000", 2024-09-16T11:44:14.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4869999999999998e-06 2024-09-16T11:44:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385609+0000", 2024-09-16T11:44:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2759999999999999e-06 2024-09-16T11:44:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413916+0000", 2024-09-16T11:44:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028307253000000001 2024-09-16T11:44:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413955+0000", 2024-09-16T11:44:14.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8652000000000002e-05 2024-09-16T11:44:14.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413989+0000", 2024-09-16T11:44:14.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3587999999999998e-05 2024-09-16T11:44:14.507 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.507 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.507 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.507 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.507 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.507 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:14.510 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:14.511 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:14.507+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:14.610 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:14.608+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:14.648 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:14.649 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:14.649 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:14.649 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:14.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:14.711 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:14.709+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:14.755 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:14.755 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:14.755 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:14.755 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:14.755 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:14.764 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:14.813 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:14.811+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:14.903 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:14.903 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:14.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:14.903 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:14.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.903 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19545 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4415488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.903 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:14.903 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.376449449999999, 2024-09-16T11:44:14.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043429581000000002, 2024-09-16T11:44:14.903 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.903 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19545 2024-09-16T11:44:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523839+0000", 2024-09-16T11:44:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0220000000000004e-06 2024-09-16T11:44:14.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523845+0000", 2024-09-16T11:44:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6500000000000001e-06 2024-09-16T11:44:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523848+0000", 2024-09-16T11:44:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1879999999999998e-06 2024-09-16T11:44:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567092+0000", 2024-09-16T11:44:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043244145999999997 2024-09-16T11:44:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567176+0000", 2024-09-16T11:44:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3825999999999993e-05 2024-09-16T11:44:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567263+0000", 2024-09-16T11:44:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7423999999999995e-05 2024-09-16T11:44:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19546 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4734976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.376431610000001, 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043599231000000002, 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19546 2024-09-16T11:44:14.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523954+0000", 2024-09-16T11:44:14.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010217099999999999 2024-09-16T11:44:14.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523957+0000", 2024-09-16T11:44:14.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3270000000000001e-06 2024-09-16T11:44:14.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523966+0000", 2024-09-16T11:44:14.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6489999999999997e-06 2024-09-16T11:44:14.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523971+0000", 2024-09-16T11:44:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567291+0000", 2024-09-16T11:44:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043319931999999998 2024-09-16T11:44:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567360+0000", 2024-09-16T11:44:14.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8628999999999996e-05 2024-09-16T11:44:14.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567451+0000", 2024-09-16T11:44:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1012000000000002e-05 2024-09-16T11:44:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19547 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4841472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.376306753000001, 2024-09-16T11:44:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043658858000000002, 2024-09-16T11:44:14.914 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19547 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525102+0000", 2024-09-16T11:44:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011257730000000001 2024-09-16T11:44:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525106+0000", 2024-09-16T11:44:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9910000000000004e-06 2024-09-16T11:44:14.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525116+0000", 2024-09-16T11:44:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0231000000000001e-05 2024-09-16T11:44:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525137+0000", 2024-09-16T11:44:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0828000000000001e-05 2024-09-16T11:44:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567478+0000", 2024-09-16T11:44:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.042340467999999999 2024-09-16T11:44:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567554+0000", 2024-09-16T11:44:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6074000000000004e-05 2024-09-16T11:44:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567635+0000", 2024-09-16T11:44:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1493000000000001e-05 2024-09-16T11:44:14.918 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19553 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4947968~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.375079691, 2024-09-16T11:44:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043450397000000002, 2024-09-16T11:44:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.919 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19553 2024-09-16T11:44:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1799999999999999e-07 2024-09-16T11:44:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525205+0000", 2024-09-16T11:44:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.062e-06 2024-09-16T11:44:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525208+0000", 2024-09-16T11:44:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8739999999999999e-06 2024-09-16T11:44:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525209+0000", 2024-09-16T11:44:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.725e-06 2024-09-16T11:44:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568520+0000", 2024-09-16T11:44:14.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043310226 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568569+0000", 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8959999999999999e-05 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568654+0000", 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5332000000000002e-05 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19554 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5267456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.375070698999998, 2024-09-16T11:44:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043659587, 2024-09-16T11:44:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19554 2024-09-16T11:44:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.924 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527103+0000", 2024-09-16T11:44:14.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018907419999999999 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527108+0000", 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6040000000000003e-06 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527118+0000", 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527122+0000", 2024-09-16T11:44:14.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9559999999999996e-06 2024-09-16T11:44:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568688+0000", 2024-09-16T11:44:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041565116999999999 2024-09-16T11:44:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568768+0000", 2024-09-16T11:44:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0474000000000002e-05 2024-09-16T11:44:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568872+0000", 2024-09-16T11:44:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104028 2024-09-16T11:44:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19565 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6332416~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.372992127, 2024-09-16T11:44:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043494890000000001, 2024-09-16T11:44:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.928 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19565 2024-09-16T11:44:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527292+0000", 2024-09-16T11:44:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.27e-07 2024-09-16T11:44:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527293+0000", 2024-09-16T11:44:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.917e-06 2024-09-16T11:44:14.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527299+0000", 2024-09-16T11:44:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6699999999999999e-06 2024-09-16T11:44:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527304+0000", 2024-09-16T11:44:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6160000000000004e-06 2024-09-16T11:44:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570683+0000", 2024-09-16T11:44:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043379171000000001 2024-09-16T11:44:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570733+0000", 2024-09-16T11:44:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0485000000000002e-05 2024-09-16T11:44:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570786+0000", 2024-09-16T11:44:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2604e-05 2024-09-16T11:44:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.932 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19566 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6758400~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.372975792999998, 2024-09-16T11:44:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043643183000000002, 2024-09-16T11:44:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19566 2024-09-16T11:44:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527308+0000", 2024-09-16T11:44:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:44:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527310+0000", 2024-09-16T11:44:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.68e-06 2024-09-16T11:44:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527316+0000", 2024-09-16T11:44:14.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9730000000000002e-06 2024-09-16T11:44:14.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527319+0000", 2024-09-16T11:44:14.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.027e-06 2024-09-16T11:44:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570810+0000", 2024-09-16T11:44:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043491525000000003 2024-09-16T11:44:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570882+0000", 2024-09-16T11:44:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:44:14.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570951+0000", 2024-09-16T11:44:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8782000000000007e-05 2024-09-16T11:44:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19567 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.372961087, 2024-09-16T11:44:14.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043786291999999997, 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19567 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529108+0000", 2024-09-16T11:44:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001785656 2024-09-16T11:44:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529113+0000", 2024-09-16T11:44:14.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1080000000000001e-06 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529122+0000", 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0550000000000005e-06 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529127+0000", 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9849999999999997e-06 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570997+0000", 2024-09-16T11:44:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041869811 2024-09-16T11:44:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571063+0000", 2024-09-16T11:44:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6340999999999994e-05 2024-09-16T11:44:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571108+0000", 2024-09-16T11:44:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5336000000000001e-05 2024-09-16T11:44:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.949 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19580 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 8036352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.370985785999999, 2024-09-16T11:44:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043451596000000002, 2024-09-16T11:44:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.950 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19580 2024-09-16T11:44:14.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.951 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.953 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:14.953 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531106+0000", 2024-09-16T11:44:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001808179 2024-09-16T11:44:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531110+0000", 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5990000000000004e-06 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531123+0000", 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2687000000000001e-05 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531129+0000", 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572642+0000", 2024-09-16T11:44:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041513474000000002 2024-09-16T11:44:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572691+0000", 2024-09-16T11:44:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8989000000000001e-05 2024-09-16T11:44:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572749+0000", 2024-09-16T11:44:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7806999999999997e-05 2024-09-16T11:44:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19708 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4530176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.368939221000002, 2024-09-16T11:44:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394863999999998, 2024-09-16T11:44:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.957 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19708 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.67e-07 2024-09-16T11:44:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531346+0000", 2024-09-16T11:44:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.477e-06 2024-09-16T11:44:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531351+0000", 2024-09-16T11:44:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1050000000000003e-06 2024-09-16T11:44:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531354+0000", 2024-09-16T11:44:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9960000000000002e-06 2024-09-16T11:44:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574620+0000", 2024-09-16T11:44:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043266365000000001 2024-09-16T11:44:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574662+0000", 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1662000000000002e-05 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574739+0000", 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6991999999999997e-05 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.961 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19709 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4849664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.368925744999999, 2024-09-16T11:44:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043537291999999998, 2024-09-16T11:44:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19709 2024-09-16T11:44:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531358+0000", 2024-09-16T11:44:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:14.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531360+0000", 2024-09-16T11:44:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:44:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531365+0000", 2024-09-16T11:44:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.716e-06 2024-09-16T11:44:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531369+0000", 2024-09-16T11:44:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5410000000000001e-06 2024-09-16T11:44:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574763+0000", 2024-09-16T11:44:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394359 2024-09-16T11:44:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574831+0000", 2024-09-16T11:44:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7997999999999995e-05 2024-09-16T11:44:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574895+0000", 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3427000000000001e-05 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19710 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4956160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.368910465999999, 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043674198999999997, 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.966 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19710 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533105+0000", 2024-09-16T11:44:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001731961 2024-09-16T11:44:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533109+0000", 2024-09-16T11:44:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3080000000000001e-06 2024-09-16T11:44:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533120+0000", 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0625000000000001e-05 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533125+0000", 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574923+0000", 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041797187 2024-09-16T11:44:14.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574991+0000", 2024-09-16T11:44:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8752999999999997e-05 2024-09-16T11:44:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.575047+0000", 2024-09-16T11:44:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5596000000000001e-05 2024-09-16T11:44:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19719 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5488640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.366938717, 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043412013999999999, 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19719 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533345+0000", 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:44:14.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533347+0000", 2024-09-16T11:44:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8139999999999999e-06 2024-09-16T11:44:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533352+0000", 2024-09-16T11:44:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4970000000000001e-06 2024-09-16T11:44:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533355+0000", 2024-09-16T11:44:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.711e-06 2024-09-16T11:44:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576629+0000", 2024-09-16T11:44:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043274295999999997 2024-09-16T11:44:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576706+0000", 2024-09-16T11:44:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6711000000000005e-05 2024-09-16T11:44:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576757+0000", 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0509e-05 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19720 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5808128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.366926068000001, 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043577006000000001, 2024-09-16T11:44:14.975 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19720 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.35e-07 2024-09-16T11:44:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533359+0000", 2024-09-16T11:44:14.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533364+0000", 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4349999999999996e-06 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533367+0000", 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2349999999999999e-06 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576786+0000", 2024-09-16T11:44:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043418885999999997 2024-09-16T11:44:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576862+0000", 2024-09-16T11:44:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6172000000000002e-05 2024-09-16T11:44:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576934+0000", 2024-09-16T11:44:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1891999999999995e-05 2024-09-16T11:44:14.979 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19721 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6234112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.366912547999998, 2024-09-16T11:44:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043681656999999999, 2024-09-16T11:44:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.980 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19721 2024-09-16T11:44:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-07 2024-09-16T11:44:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533373+0000", 2024-09-16T11:44:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.973e-06 2024-09-16T11:44:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533377+0000", 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.403e-06 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533379+0000", 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.058e-06 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576959+0000", 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043579433000000001 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577015+0000", 2024-09-16T11:44:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5859000000000002e-05 2024-09-16T11:44:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577052+0000", 2024-09-16T11:44:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7620999999999998e-05 2024-09-16T11:44:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.984 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19722 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6340608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.366901820999999, 2024-09-16T11:44:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043854922999999997, 2024-09-16T11:44:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19722 2024-09-16T11:44:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535103+0000", 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017217020000000001 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535107+0000", 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2819999999999998e-06 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535118+0000", 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0148e-05 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535123+0000", 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:44:14.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577091+0000", 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041967930000000001 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577163+0000", 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1853999999999993e-05 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577236+0000", 2024-09-16T11:44:14.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3699e-05 2024-09-16T11:44:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19732 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6447104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.365041965, 2024-09-16T11:44:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043399048000000003, 2024-09-16T11:44:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.989 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19732 2024-09-16T11:44:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:14.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535242+0000", 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535243+0000", 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.793e-06 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535249+0000", 2024-09-16T11:44:14.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.643e-06 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535252+0000", 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578501+0000", 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043248343000000002 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578576+0000", 2024-09-16T11:44:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5544999999999996e-05 2024-09-16T11:44:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578640+0000", 2024-09-16T11:44:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4018999999999998e-05 2024-09-16T11:44:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19733 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6873088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.365021123999998, 2024-09-16T11:44:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043549681, 2024-09-16T11:44:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19733 2024-09-16T11:44:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:14.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535263+0000", 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535265+0000", 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9369999999999998e-06 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535271+0000", 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9719999999999997e-06 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535274+0000", 2024-09-16T11:44:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:44:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578668+0000", 2024-09-16T11:44:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043393355000000002 2024-09-16T11:44:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578744+0000", 2024-09-16T11:44:14.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6853000000000004e-05 2024-09-16T11:44:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578812+0000", 2024-09-16T11:44:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.737e-05 2024-09-16T11:44:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19734 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7299072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:14.998 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.365005376999999, 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043699189999999999, 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19734 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:14.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537101+0000", 2024-09-16T11:44:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001823098 2024-09-16T11:44:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537105+0000", 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537119+0000", 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614999999999999e-05 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537125+0000", 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578838+0000", 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041712605999999999 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578912+0000", 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4396999999999999e-05 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578977+0000", 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4858999999999997e-05 2024-09-16T11:44:15.002 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19851 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4431872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 19.362924632999999, 2024-09-16T11:44:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043564540999999998, 2024-09-16T11:44:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:15.003 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19851 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539100+0000", 2024-09-16T11:44:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017412079999999999 2024-09-16T11:44:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539105+0000", 2024-09-16T11:44:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T11:44:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539116+0000", 2024-09-16T11:44:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1824999999999999e-05 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539122+0000", 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9859999999999999e-06 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580764+0000", 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041641299 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:15.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580839+0000", 2024-09-16T11:44:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5167999999999996e-05 2024-09-16T11:44:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580923+0000", 2024-09-16T11:44:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4314999999999995e-05 2024-09-16T11:44:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:15.007 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:15.007 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:15.008 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:44:15.009 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:15.006+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:15.010 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:15.010 INFO:teuthology.orchestra.run.smithi070.stdout:flattening image image_client.0-clone26 2024-09-16T11:44:15.108 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:15.108 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:15.108 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:15.108 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:15.109 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:15.107+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:15.117 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:15.150 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-16T11:44:15.153 INFO:teuthology.orchestra.run.smithi070.stdout:4319 write 0x67c6ee2 thru 0x67d1c06 (0xad25 bytes) 2024-09-16T11:44:15.160 INFO:teuthology.orchestra.run.smithi070.stdout:4320 write 0x9277963 thru 0x927f394 (0x7a32 bytes) 2024-09-16T11:44:15.167 INFO:teuthology.orchestra.run.smithi070.stdout:4321 trunc from 0xc670478 to 0x91063ac 2024-09-16T11:44:15.178 INFO:teuthology.orchestra.run.smithi070.stdout:4322 read 0x3927c33 thru 0x3934820 (0xcbee bytes) 2024-09-16T11:44:15.178 INFO:teuthology.orchestra.run.smithi070.stdout:4324 read 0x371fb2 thru 0x37fc4e (0xdc9d bytes) 2024-09-16T11:44:15.178 INFO:teuthology.orchestra.run.smithi070.stdout:4325 write 0x72e5c74 thru 0x72e8af6 (0x2e83 bytes) 2024-09-16T11:44:15.181 INFO:teuthology.orchestra.run.smithi070.stdout:4327 write 0x88d1539 thru 0x88e04d0 (0xef98 bytes) 2024-09-16T11:44:15.184 INFO:teuthology.orchestra.run.smithi070.stdout:4328 read 0x7a8f7c7 thru 0x7a9430d (0x4b47 bytes) 2024-09-16T11:44:15.184 INFO:teuthology.orchestra.run.smithi070.stdout:4330 read 0x7312b68 thru 0x732082b (0xdcc4 bytes) 2024-09-16T11:44:15.184 INFO:teuthology.orchestra.run.smithi070.stdout:4331 read 0x5b9e289 thru 0x5ba6fcd (0x8d45 bytes) 2024-09-16T11:44:15.193 INFO:teuthology.orchestra.run.smithi070.stdout:4332 trunc from 0x91063ac to 0x1d31d5e 2024-09-16T11:44:15.210 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:15.207+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:15.210 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:15.224 INFO:teuthology.orchestra.run.smithi070.stdout:4334 trunc from 0x1d31d5e to 0xd8a7119 2024-09-16T11:44:15.229 INFO:teuthology.orchestra.run.smithi070.stdout:4336 write 0xbd1d1f9 thru 0xbd27e20 (0xac28 bytes) 2024-09-16T11:44:15.232 INFO:teuthology.orchestra.run.smithi070.stdout:4337 clone 28 order 25 su 4194304 sc 15 2024-09-16T11:44:15.259 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:15.259 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:15.259 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:15.259 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:15.259 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:15.267 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:15.350 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:15.348+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:15.407 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042994207, 2024-09-16T11:44:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:44:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:44:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:44:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:44:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042972177999999, 2024-09-16T11:44:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:44:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:44:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:44:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:44:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:44:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:44:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:44:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:44:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:44:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042957413, 2024-09-16T11:44:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:44:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:44:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:44:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:44:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:44:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:44:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:44:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:44:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:44:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:44:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:44:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042937344999999, 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:44:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:44:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:44:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:44:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:44:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:44:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:44:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:44:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:44:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:44:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:44:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:44:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:44:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042906509000002, 2024-09-16T11:44:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:44:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:44:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:44:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:44:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:44:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:44:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:44:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042886988999999, 2024-09-16T11:44:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:44:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:44:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:44:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:44:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:44:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:44:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:44:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:44:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042859409999998, 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:44:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:44:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:44:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:44:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:44:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:44:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:44:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042840715000001, 2024-09-16T11:44:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:44:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:44:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:44:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:44:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:44:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:44:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:44:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:44:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:44:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:44:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:44:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:44:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042825705999999, 2024-09-16T11:44:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:44:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:44:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042793608, 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:44:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:44:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042769989, 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:44:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.04275526, 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:44:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:44:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:44:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:44:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042733435999999, 2024-09-16T11:44:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:44:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:44:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042718785999998, 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042697213, 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:44:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042677767000001, 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:44:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:44:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:44:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:44:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:44:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:44:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:44:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042641069999998, 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:44:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:44:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:44:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:44:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:44:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:44:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:44:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042619124000002, 2024-09-16T11:44:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:44:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:44:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:44:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:44:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:44:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:44:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:44:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042597237999999, 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:44:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:44:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:44:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:44:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:44:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:44:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:44:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:44:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.042582519, 2024-09-16T11:44:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:44:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:44:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:44:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:44:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.496 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:15.498 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:15.499 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:15.496+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:15.599 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:15.596+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:15.636 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:15.636 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:15.637 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:15.637 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:15.645 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:15.729 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:15.727+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:15.784 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:15.785 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:15.793 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:15.820 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:44:15.835 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:15.832+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:15.901 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:15.899+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:15.934 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:15.934 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:15.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:15.934 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:15.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.935 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19942 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1990656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.935 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:15.935 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405480776000001, 2024-09-16T11:44:15.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027866354999999999, 2024-09-16T11:44:15.935 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.935 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.935 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.935 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.935 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.935 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19942 2024-09-16T11:44:15.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.935 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:15.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:15.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:15.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4899999999999995e-07 2024-09-16T11:44:15.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526281+0000", 2024-09-16T11:44:15.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-16T11:44:15.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526287+0000", 2024-09-16T11:44:15.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2199999999999997e-06 2024-09-16T11:44:15.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526291+0000", 2024-09-16T11:44:15.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:44:15.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554068+0000", 2024-09-16T11:44:15.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027777076000000001 2024-09-16T11:44:15.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554104+0000", 2024-09-16T11:44:15.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5537000000000002e-05 2024-09-16T11:44:15.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554145+0000", 2024-09-16T11:44:15.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1350000000000002e-05 2024-09-16T11:44:15.940 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.940 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.940 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.940 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19943 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2416640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.940 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:15.941 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405464472999999, 2024-09-16T11:44:15.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027947257999999999, 2024-09-16T11:44:15.941 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.941 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.941 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19943 2024-09-16T11:44:15.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.941 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:15.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:15.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:15.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5999999999999999e-07 2024-09-16T11:44:15.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526297+0000", 2024-09-16T11:44:15.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8059999999999999e-06 2024-09-16T11:44:15.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526303+0000", 2024-09-16T11:44:15.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6579999999999999e-06 2024-09-16T11:44:15.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526306+0000", 2024-09-16T11:44:15.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3969999999999998e-06 2024-09-16T11:44:15.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554165+0000", 2024-09-16T11:44:15.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027858519000000002 2024-09-16T11:44:15.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554201+0000", 2024-09-16T11:44:15.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5908000000000002e-05 2024-09-16T11:44:15.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554242+0000", 2024-09-16T11:44:15.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1510000000000001e-05 2024-09-16T11:44:15.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.945 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.945 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19944 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2842624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.946 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:15.946 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405449577999999, 2024-09-16T11:44:15.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028052639000000001, 2024-09-16T11:44:15.946 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.946 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.946 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19944 2024-09-16T11:44:15.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.946 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:15.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:15.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:15.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:44:15.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526312+0000", 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526317+0000", 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5910000000000003e-06 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526321+0000", 2024-09-16T11:44:15.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2339999999999999e-06 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554263+0000", 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027942181 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554307+0000", 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4499999999999997e-05 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554362+0000", 2024-09-16T11:44:15.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5078999999999999e-05 2024-09-16T11:44:15.950 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.950 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.950 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.950 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19945 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3162112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.950 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:15.950 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405435022999999, 2024-09-16T11:44:15.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028145673999999999, 2024-09-16T11:44:15.950 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.950 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.950 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19945 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:15.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526325+0000", 2024-09-16T11:44:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:44:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526326+0000", 2024-09-16T11:44:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.733e-06 2024-09-16T11:44:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526332+0000", 2024-09-16T11:44:15.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:44:15.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526336+0000", 2024-09-16T11:44:15.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8410000000000002e-06 2024-09-16T11:44:15.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554389+0000", 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028053576 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554427+0000", 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7299999999999999e-05 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554470+0000", 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3195000000000002e-05 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.954 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19946 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3268608~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405419444, 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028232587, 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19946 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.955 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:15.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:15.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526346+0000", 2024-09-16T11:44:15.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6459999999999998e-06 2024-09-16T11:44:15.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526352+0000", 2024-09-16T11:44:15.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1070000000000001e-06 2024-09-16T11:44:15.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526355+0000", 2024-09-16T11:44:15.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:44:15.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554493+0000", 2024-09-16T11:44:15.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.02813746 2024-09-16T11:44:15.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554530+0000", 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6881999999999997e-05 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554572+0000", 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2688999999999997e-05 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.959 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19947 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4014080~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.960 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:15.960 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405400090000001, 2024-09-16T11:44:15.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028315646999999999, 2024-09-16T11:44:15.960 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.960 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.960 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.960 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.960 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.960 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19947 2024-09-16T11:44:15.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.961 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:15.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:15.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526360+0000", 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.01e-07 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526361+0000", 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8300000000000001e-06 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526369+0000", 2024-09-16T11:44:15.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526377+0000", 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0909999999999993e-06 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554593+0000", 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028216657999999999 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554632+0000", 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8828000000000002e-05 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554675+0000", 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2710000000000003e-05 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19948 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3588096~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:15.964 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405377690999998, 2024-09-16T11:44:15.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028394545, 2024-09-16T11:44:15.965 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.965 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.965 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19948 2024-09-16T11:44:15.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.965 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:15.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:15.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:15.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9200000000000001e-07 2024-09-16T11:44:15.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526384+0000", 2024-09-16T11:44:15.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:44:15.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526391+0000", 2024-09-16T11:44:15.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5150000000000003e-06 2024-09-16T11:44:15.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526394+0000", 2024-09-16T11:44:15.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4240000000000002e-06 2024-09-16T11:44:15.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554695+0000", 2024-09-16T11:44:15.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028301218999999999 2024-09-16T11:44:15.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554731+0000", 2024-09-16T11:44:15.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6298999999999999e-05 2024-09-16T11:44:15.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554776+0000", 2024-09-16T11:44:15.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4675999999999997e-05 2024-09-16T11:44:15.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.969 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.970 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.970 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19949 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1351680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.970 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:15.970 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405361418999998, 2024-09-16T11:44:15.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028490081, 2024-09-16T11:44:15.970 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.970 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.970 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19949 2024-09-16T11:44:15.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.971 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:15.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:15.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:15.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:44:15.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526400+0000", 2024-09-16T11:44:15.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7099999999999999e-06 2024-09-16T11:44:15.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526406+0000", 2024-09-16T11:44:15.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5140000000000001e-06 2024-09-16T11:44:15.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526409+0000", 2024-09-16T11:44:15.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5360000000000001e-06 2024-09-16T11:44:15.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554801+0000", 2024-09-16T11:44:15.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028392378999999999 2024-09-16T11:44:15.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554843+0000", 2024-09-16T11:44:15.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1056000000000001e-05 2024-09-16T11:44:15.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554888+0000", 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466999999999997e-05 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19950 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1777664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405346423000001, 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028661844999999998, 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.975 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19950 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:15.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526415+0000", 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.776e-06 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526422+0000", 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6930000000000001e-06 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526425+0000", 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554923+0000", 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028498102000000001 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555000+0000", 2024-09-16T11:44:15.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6619999999999995e-05 2024-09-16T11:44:15.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555075+0000", 2024-09-16T11:44:15.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4931999999999993e-05 2024-09-16T11:44:15.979 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.979 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.979 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.980 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19951 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2097152~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.980 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:15.980 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405330612, 2024-09-16T11:44:15.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028755718, 2024-09-16T11:44:15.980 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.980 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.980 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.980 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.980 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.980 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19951 2024-09-16T11:44:15.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.980 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:15.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:15.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:15.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526431+0000", 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526441+0000", 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7790000000000007e-06 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526445+0000", 2024-09-16T11:44:15.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.634e-06 2024-09-16T11:44:15.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555100+0000", 2024-09-16T11:44:15.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654722000000001 2024-09-16T11:44:15.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555138+0000", 2024-09-16T11:44:15.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7996e-05 2024-09-16T11:44:15.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555184+0000", 2024-09-16T11:44:15.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6427999999999997e-05 2024-09-16T11:44:15.984 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.984 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.985 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.985 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19952 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2203648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.985 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:15.985 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405310052000001, 2024-09-16T11:44:15.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028901157, 2024-09-16T11:44:15.985 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.985 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.986 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19952 2024-09-16T11:44:15.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.986 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:15.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526450+0000", 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526452+0000", 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9039999999999999e-06 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526465+0000", 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3559e-05 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526470+0000", 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555217+0000", 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028747287999999999 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555272+0000", 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4885000000000001e-05 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555350+0000", 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8428000000000002e-05 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19953 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2523136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.989 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:15.990 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405285460000002, 2024-09-16T11:44:15.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029004309999999998, 2024-09-16T11:44:15.990 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.990 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.990 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19953 2024-09-16T11:44:15.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.990 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:15.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:15.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:15.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-16T11:44:15.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526476+0000", 2024-09-16T11:44:15.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9810000000000002e-06 2024-09-16T11:44:15.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526482+0000", 2024-09-16T11:44:15.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:15.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526485+0000", 2024-09-16T11:44:15.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2760000000000001e-06 2024-09-16T11:44:15.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555383+0000", 2024-09-16T11:44:15.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028898113 2024-09-16T11:44:15.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555433+0000", 2024-09-16T11:44:15.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9753999999999998e-05 2024-09-16T11:44:15.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555478+0000", 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5006999999999998e-05 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19954 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2949120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405270463000001, 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029165054999999999, 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19954 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:15.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1799999999999995e-07 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526491+0000", 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7320000000000001e-06 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526497+0000", 2024-09-16T11:44:15.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6869999999999999e-06 2024-09-16T11:44:15.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:15.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526500+0000", 2024-09-16T11:44:15.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:44:15.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:15.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555513+0000", 2024-09-16T11:44:15.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029012626 2024-09-16T11:44:15.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:15.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555576+0000", 2024-09-16T11:44:15.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3288999999999997e-05 2024-09-16T11:44:15.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:15.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555654+0000", 2024-09-16T11:44:15.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7861999999999994e-05 2024-09-16T11:44:15.998 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.998 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:15.998 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:15.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:15.998 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19955 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3375104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:15.999 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:15.999 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405251160999999, 2024-09-16T11:44:15.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029241979000000001, 2024-09-16T11:44:15.999 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:15.999 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:15.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:15.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:15.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:15.999 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19955 2024-09-16T11:44:15.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:15.999 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:15.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:16.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:16.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:16.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:16.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:16.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526509+0000", 2024-09-16T11:44:16.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526510+0000", 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.844e-06 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526516+0000", 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9039999999999997e-06 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:16.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526520+0000", 2024-09-16T11:44:16.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3639999999999999e-06 2024-09-16T11:44:16.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:16.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555669+0000", 2024-09-16T11:44:16.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029149628 2024-09-16T11:44:16.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:16.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555706+0000", 2024-09-16T11:44:16.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7017000000000002e-05 2024-09-16T11:44:16.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:16.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555750+0000", 2024-09-16T11:44:16.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3856999999999997e-05 2024-09-16T11:44:16.003 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.003 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:16.003 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.003 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19956 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3694592~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:16.003 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:16.003 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405236206000001, 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029398799999999999, 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19956 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:16.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:16.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:16.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526524+0000", 2024-09-16T11:44:16.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:44:16.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:16.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526525+0000", 2024-09-16T11:44:16.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-16T11:44:16.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:16.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526531+0000", 2024-09-16T11:44:16.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6010000000000002e-06 2024-09-16T11:44:16.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:16.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526535+0000", 2024-09-16T11:44:16.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.597e-06 2024-09-16T11:44:16.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:16.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555785+0000", 2024-09-16T11:44:16.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029250235999999999 2024-09-16T11:44:16.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:16.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555845+0000", 2024-09-16T11:44:16.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0393000000000003e-05 2024-09-16T11:44:16.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:16.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555922+0000", 2024-09-16T11:44:16.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6710000000000002e-05 2024-09-16T11:44:16.008 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.008 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:16.008 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.009 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19957 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4120576~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:16.009 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:16.009 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.405221052999998, 2024-09-16T11:44:16.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029481157000000001, 2024-09-16T11:44:16.010 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:16.010 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:16.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:16.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:16.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:16.010 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19957 2024-09-16T11:44:16.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.010 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:16.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:16.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:16.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:16.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:16.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:16.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526539+0000", 2024-09-16T11:44:16.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:16.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:16.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526540+0000", 2024-09-16T11:44:16.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.759e-06 2024-09-16T11:44:16.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:16.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526546+0000", 2024-09-16T11:44:16.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1079999999999998e-06 2024-09-16T11:44:16.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:16.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526550+0000", 2024-09-16T11:44:16.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.269e-06 2024-09-16T11:44:16.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:16.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555939+0000", 2024-09-16T11:44:16.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029389444000000001 2024-09-16T11:44:16.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:16.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555977+0000", 2024-09-16T11:44:16.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7487999999999998e-05 2024-09-16T11:44:16.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:16.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.556019+0000", 2024-09-16T11:44:16.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2756999999999997e-05 2024-09-16T11:44:16.014 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.014 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:16.014 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.014 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:28959 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:16.014 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:16.014 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.922364641, 2024-09-16T11:44:16.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031145285000000002, 2024-09-16T11:44:16.014 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:16.015 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:16.015 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:16.015 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:16.015 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:16.015 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 28959 2024-09-16T11:44:16.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.015 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:16.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:16.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:16.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:16.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:16.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:16.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009396+0000", 2024-09-16T11:44:16.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.296e-06 2024-09-16T11:44:16.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:16.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030864+0000", 2024-09-16T11:44:16.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.021468451 2024-09-16T11:44:16.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:16.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030892+0000", 2024-09-16T11:44:16.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7555999999999998e-05 2024-09-16T11:44:16.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:16.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030902+0000", 2024-09-16T11:44:16.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9550000000000004e-06 2024-09-16T11:44:16.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030979+0000", 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7137000000000004e-05 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031051+0000", 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1475000000000003e-05 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031389+0000", 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033827100000000001 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:16.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.039264+0000", 2024-09-16T11:44:16.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078751700000000008 2024-09-16T11:44:16.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:16.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040484+0000", 2024-09-16T11:44:16.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012199579999999999 2024-09-16T11:44:16.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:16.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040517+0000", 2024-09-16T11:44:16.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2960999999999998e-05 2024-09-16T11:44:16.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:16.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040540+0000", 2024-09-16T11:44:16.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3054999999999999e-05 2024-09-16T11:44:16.020 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.021 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:16.021 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.021 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30605 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 3915776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:16.021 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:16.021 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.688058381999999, 2024-09-16T11:44:16.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938655999999999, 2024-09-16T11:44:16.021 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:16.021 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:16.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30605 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:16.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243704+0000", 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2199999999999999e-06 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:16.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243710+0000", 2024-09-16T11:44:16.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6340000000000003e-06 2024-09-16T11:44:16.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:16.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243714+0000", 2024-09-16T11:44:16.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5030000000000002e-06 2024-09-16T11:44:16.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:16.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271450+0000", 2024-09-16T11:44:16.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027736035999999999 2024-09-16T11:44:16.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:16.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271536+0000", 2024-09-16T11:44:16.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6521999999999996e-05 2024-09-16T11:44:16.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:16.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271640+0000", 2024-09-16T11:44:16.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103341 2024-09-16T11:44:16.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.025 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:16.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.026 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30607 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4046848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:16.026 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:16.026 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.688035227, 2024-09-16T11:44:16.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028130446, 2024-09-16T11:44:16.026 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:16.026 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:16.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:16.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:16.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:16.026 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30607 2024-09-16T11:44:16.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.027 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:16.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:16.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:16.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:16.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:16.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:16.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243725+0000", 2024-09-16T11:44:16.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2399999999999998e-07 2024-09-16T11:44:16.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:16.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243727+0000", 2024-09-16T11:44:16.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3379999999999999e-06 2024-09-16T11:44:16.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:16.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243733+0000", 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4280000000000001e-06 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243737+0000", 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8990000000000003e-06 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271676+0000", 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938481000000001 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:16.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271757+0000", 2024-09-16T11:44:16.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1396999999999993e-05 2024-09-16T11:44:16.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:16.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271855+0000", 2024-09-16T11:44:16.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7379000000000004e-05 2024-09-16T11:44:16.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.030 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:16.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.031 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30608 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:16.031 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:16.031 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.688018376, 2024-09-16T11:44:16.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028323879, 2024-09-16T11:44:16.031 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:16.031 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:16.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:16.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:16.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:16.031 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30608 2024-09-16T11:44:16.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.031 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:16.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:16.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:16.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:16.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:16.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:16.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:16.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:16.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:44:16.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:16.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243743+0000", 2024-09-16T11:44:16.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.762e-06 2024-09-16T11:44:16.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:16.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243749+0000", 2024-09-16T11:44:16.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5670000000000002e-06 2024-09-16T11:44:16.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:16.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243758+0000", 2024-09-16T11:44:16.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8309999999999996e-06 2024-09-16T11:44:16.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:16.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271879+0000", 2024-09-16T11:44:16.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028120501999999999 2024-09-16T11:44:16.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:16.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271965+0000", 2024-09-16T11:44:16.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5755999999999997e-05 2024-09-16T11:44:16.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:16.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:16.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:16.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.272065+0000", 2024-09-16T11:44:16.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000100101 2024-09-16T11:44:16.035 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.035 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:16.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:16.036 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:16.036 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:16.037 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:16.037 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:16.038 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T11:44:16.038 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:16.039 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:16.039 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:16.040 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:16.040 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:16.041 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:16.041 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:16.041 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone27 from 0xd8a7119 (overlap 0x1d31d5e) to 0xbad88c 2024-09-16T11:44:16.042 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:16.042 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:16.042 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:16.042 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:16.043 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:16.035+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:16.043 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:16.043 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:16.054 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:16.055 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-16T11:44:16.138 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:16.136+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:16.177 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:16.178 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:16.186 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:16.239 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:16.237+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:16.270 INFO:teuthology.orchestra.run.smithi070.stdout:4338 write 0x3aa4b03 thru 0x3aae803 (0x9d01 bytes) 2024-09-16T11:44:16.277 INFO:teuthology.orchestra.run.smithi070.stdout:4340 read 0x3cedc07 thru 0x3cf595d (0x7d57 bytes) 2024-09-16T11:44:16.277 INFO:teuthology.orchestra.run.smithi070.stdout:4341 write 0x275884c thru 0x275c49a (0x3c4f bytes) 2024-09-16T11:44:16.280 INFO:teuthology.orchestra.run.smithi070.stdout:4345 write 0x3f8b403 thru 0x3f8e8af (0x34ad bytes) 2024-09-16T11:44:16.285 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:16.285 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:16.294 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:16.298 INFO:teuthology.orchestra.run.smithi070.stdout:4346 write 0xdc06a88 thru 0xdc1275a (0xbcd3 bytes) 2024-09-16T11:44:16.301 INFO:teuthology.orchestra.run.smithi070.stdout:4347 punch from 0xd392733 to 0xd398ffd, (0x68ca bytes) 2024-09-16T11:44:16.301 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.301 INFO:teuthology.orchestra.run.smithi070.stdout:4349 punch from 0x50d757e to 0x50d8ddf, (0x1861 bytes) 2024-09-16T11:44:16.301 INFO:teuthology.orchestra.run.smithi070.stdout:4350 write 0xbd2d62b thru 0xbd2e752 (0x1128 bytes) 2024-09-16T11:44:16.317 INFO:teuthology.orchestra.run.smithi070.stdout:4351 read 0x3501e15 thru 0x350c87b (0xaa67 bytes) 2024-09-16T11:44:16.317 INFO:teuthology.orchestra.run.smithi070.stdout:4352 read 0x4906822 thru 0x490ca88 (0x6267 bytes) 2024-09-16T11:44:16.317 INFO:teuthology.orchestra.run.smithi070.stdout:4353 flatten 2024-09-16T11:44:16.325 INFO:teuthology.orchestra.run.smithi070.stdout:4354 read 0xbb0cc49 thru 0xbb0d2f2 (0x6aa bytes) 2024-09-16T11:44:16.325 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.340 INFO:teuthology.orchestra.run.smithi070.stdout:4356 trunc from 0xdc1275b to 0xf38ee2 2024-09-16T11:44:16.340 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:16.338+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:16.357 INFO:teuthology.orchestra.run.smithi070.stdout:4357 writesame 0x16e55cf thru 0x16eb23d data_size 0x1(0x5c6f bytes) 2024-09-16T11:44:16.358 INFO:teuthology.orchestra.run.smithi070.stdout:4358 punch from 0x3e9df6 to 0x3ea216, (0x420 bytes) 2024-09-16T11:44:16.363 INFO:teuthology.orchestra.run.smithi070.stdout:4359 write 0x220e6d2 thru 0x220f9ff (0x132e bytes) 2024-09-16T11:44:16.366 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.367 INFO:teuthology.orchestra.run.smithi070.stdout:4362 trunc from 0x220fa00 to 0x19cb127 2024-09-16T11:44:16.385 INFO:teuthology.orchestra.run.smithi070.stdout:4364 trunc from 0x19cb127 to 0xa1d5eb4 2024-09-16T11:44:16.391 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.391 INFO:teuthology.orchestra.run.smithi070.stdout:4367 punch from 0x80b8291 to 0x80bd9d6, (0x5745 bytes) 2024-09-16T11:44:16.391 INFO:teuthology.orchestra.run.smithi070.stdout:4368 write 0x12266eb thru 0x1227bb5 (0x14cb bytes) 2024-09-16T11:44:16.394 INFO:teuthology.orchestra.run.smithi070.stdout:4369 read 0x6648225 thru 0x665324f (0xb02b bytes) 2024-09-16T11:44:16.394 INFO:teuthology.orchestra.run.smithi070.stdout:4370 writesame 0x5d1afb0 thru 0x5d1eb7c data_size 0x9(0x3bcd bytes) 2024-09-16T11:44:16.397 INFO:teuthology.orchestra.run.smithi070.stdout:4371 read 0x26a8378 thru 0x26b63c2 (0xe04b bytes) 2024-09-16T11:44:16.399 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:16.400 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:16.408 INFO:teuthology.orchestra.run.smithi070.stdout:4372 trunc from 0xa1d5eb4 to 0xc5a2d4 2024-09-16T11:44:16.409 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:16.419 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.441 INFO:teuthology.orchestra.run.smithi070.stdout:4375 writesame 0xe7a50f0 thru 0xe7a69ac data_size 0x1(0x18bd bytes) 2024-09-16T11:44:16.441 INFO:teuthology.orchestra.run.smithi070.stdout:4377 punch from 0xbc41a92 to 0xbc458aa, (0x3e18 bytes) 2024-09-16T11:44:16.441 INFO:teuthology.orchestra.run.smithi070.stdout:4379 punch from 0xe42ca38 to 0xe42da07, (0xfcf bytes) 2024-09-16T11:44:16.441 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.441 INFO:teuthology.orchestra.run.smithi070.stdout:4381 read 0xb751761 thru 0xb754cb4 (0x3554 bytes) 2024-09-16T11:44:16.441 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.441 INFO:teuthology.orchestra.run.smithi070.stdout:4383 write 0x509b2c1 thru 0x50a988e (0xe5ce bytes) 2024-09-16T11:44:16.444 INFO:teuthology.orchestra.run.smithi070.stdout:4384 writesame 0x716d7ef thru 0x717cc52 data_size 0x2(0xf464 bytes) 2024-09-16T11:44:16.452 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:16.450+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:16.460 INFO:teuthology.orchestra.run.smithi070.stdout:4385 read 0x683077b thru 0x683db9f (0xd425 bytes) 2024-09-16T11:44:16.460 INFO:teuthology.orchestra.run.smithi070.stdout:4388 punch from 0xc6fcd05 to 0xc702f1f, (0x621a bytes) 2024-09-16T11:44:16.460 INFO:teuthology.orchestra.run.smithi070.stdout:4389 write 0xb89094e thru 0xb894530 (0x3be3 bytes) 2024-09-16T11:44:16.476 INFO:teuthology.orchestra.run.smithi070.stdout:4390 trunc from 0xe7a69ad to 0x36648a0 2024-09-16T11:44:16.483 INFO:teuthology.orchestra.run.smithi070.stdout:4391 punch from 0x1bd9b2d to 0x1be2f50, (0x9423 bytes) 2024-09-16T11:44:16.493 INFO:teuthology.orchestra.run.smithi070.stdout:4392 trunc from 0x36648a0 to 0xc33310c 2024-09-16T11:44:16.499 INFO:teuthology.orchestra.run.smithi070.stdout:4394 read 0x63d2932 thru 0x63e18c5 (0xef94 bytes) 2024-09-16T11:44:16.499 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.507 INFO:teuthology.orchestra.run.smithi070.stdout:4397 writesame 0xebe62df thru 0xebeb978 data_size 0xf(0x569a bytes) 2024-09-16T11:44:16.509 INFO:teuthology.orchestra.run.smithi070.stdout:4398 write 0x9bc7344 thru 0x9bd55f9 (0xe2b6 bytes) 2024-09-16T11:44:16.510 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:16.510 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:16.512 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.512 INFO:teuthology.orchestra.run.smithi070.stdout:4400 read 0x7b2b205 thru 0x7b353c1 (0xa1bd bytes) 2024-09-16T11:44:16.512 INFO:teuthology.orchestra.run.smithi070.stdout:4401 punch from 0x3b29d75 to 0x3b38b92, (0xee1d bytes) 2024-09-16T11:44:16.513 INFO:teuthology.orchestra.run.smithi070.stdout:4402 write 0x9c33535 thru 0x9c38826 (0x52f2 bytes) 2024-09-16T11:44:16.516 INFO:teuthology.orchestra.run.smithi070.stdout:4403 read 0x1e27754 thru 0x1e32009 (0xa8b6 bytes) 2024-09-16T11:44:16.517 INFO:teuthology.orchestra.run.smithi070.stdout:4404 writesame 0xcd8f5d0 thru 0xcd9bdaa data_size 0x1(0xc7db bytes) 2024-09-16T11:44:16.519 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:16.541 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:16.539+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:16.542 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.542 INFO:teuthology.orchestra.run.smithi070.stdout:4406 writesame 0x7c0fd39 thru 0x7c123c4 data_size 0x2(0x268c bytes) 2024-09-16T11:44:16.552 INFO:teuthology.orchestra.run.smithi070.stdout:4407 trunc from 0xebeb979 to 0x82e0acf 2024-09-16T11:44:16.559 INFO:teuthology.orchestra.run.smithi070.stdout:4408 read 0x2e6b1cd thru 0x2e79054 (0xde88 bytes) 2024-09-16T11:44:16.568 INFO:teuthology.orchestra.run.smithi070.stdout:4409 trunc from 0x82e0acf to 0x31710e 2024-09-16T11:44:16.594 INFO:teuthology.orchestra.run.smithi070.stdout:4411 write 0x297900c thru 0x2984c09 (0xbbfe bytes) 2024-09-16T11:44:16.597 INFO:teuthology.orchestra.run.smithi070.stdout:4412 punch from 0x289bda5 to 0x28a1491, (0x56ec bytes) 2024-09-16T11:44:16.597 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.597 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.604 INFO:teuthology.orchestra.run.smithi070.stdout:4415 writesame 0x38555ed thru 0x3862d6a data_size 0x1(0xd77e bytes) 2024-09-16T11:44:16.604 INFO:teuthology.orchestra.run.smithi070.stdout:4416 read 0x350a6b4 thru 0x3517616 (0xcf63 bytes) 2024-09-16T11:44:16.604 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.604 INFO:teuthology.orchestra.run.smithi070.stdout:4418 read 0x1f54082 thru 0x1f636b0 (0xf62f bytes) 2024-09-16T11:44:16.604 INFO:teuthology.orchestra.run.smithi070.stdout:4419 read 0x2bfe19 thru 0x2c1da4 (0x1f8c bytes) 2024-09-16T11:44:16.612 INFO:teuthology.orchestra.run.smithi070.stdout:4420 writesame 0x5eb9c75 thru 0x5ec6922 data_size 0x7b(0xccae bytes) 2024-09-16T11:44:16.621 INFO:teuthology.orchestra.run.smithi070.stdout:4421 trunc from 0x5ec6923 to 0xd30052b 2024-09-16T11:44:16.627 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.627 INFO:teuthology.orchestra.run.smithi070.stdout:4424 write 0x7a1c6ac thru 0x7a28621 (0xbf76 bytes) 2024-09-16T11:44:16.634 INFO:teuthology.orchestra.run.smithi070.stdout:4425 writesame 0xd54a0b1 thru 0xd5560bd data_size 0x1(0xc00d bytes) 2024-09-16T11:44:16.640 INFO:teuthology.orchestra.run.smithi070.stdout:4426 trunc from 0xd5560be to 0x81f33ca 2024-09-16T11:44:16.641 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:16.639+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:16.647 INFO:teuthology.orchestra.run.smithi070.stdout:4427 read 0x79d8418 thru 0x79e1503 (0x90ec bytes) 2024-09-16T11:44:16.647 INFO:teuthology.orchestra.run.smithi070.stdout:4428 write 0x7790718 thru 0x77953f3 (0x4cdc bytes) 2024-09-16T11:44:16.652 INFO:teuthology.orchestra.run.smithi070.stdout:4429 write 0x64b7b60 thru 0x64bde30 (0x62d1 bytes) 2024-09-16T11:44:16.659 INFO:teuthology.orchestra.run.smithi070.stdout:4431 trunc from 0x81f33ca to 0x1b6af01 2024-09-16T11:44:16.664 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:16.664 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:16.671 INFO:teuthology.orchestra.run.smithi070.stdout:4433 trunc from 0x1b6af01 to 0x4f399b 2024-09-16T11:44:16.673 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:16.676 INFO:teuthology.orchestra.run.smithi070.stdout:4434 punch from 0x182054 to 0x1917c1, (0xf76d bytes) 2024-09-16T11:44:16.676 INFO:teuthology.orchestra.run.smithi070.stdout:4437 read 0x2047cc thru 0x2056d5 (0xf0a bytes) 2024-09-16T11:44:16.677 INFO:teuthology.orchestra.run.smithi070.stdout:4439 read 0xae991 thru 0xbb424 (0xca94 bytes) 2024-09-16T11:44:16.677 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.677 INFO:teuthology.orchestra.run.smithi070.stdout:4442 read 0x17c15d thru 0x18b79f (0xf643 bytes) 2024-09-16T11:44:16.678 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.683 INFO:teuthology.orchestra.run.smithi070.stdout:4446 write 0x8cf8e9 thru 0x8d8525 (0x8c3d bytes) 2024-09-16T11:44:16.685 INFO:teuthology.orchestra.run.smithi070.stdout:4447 punch from 0x7c0231 to 0x7c8e43, (0x8c12 bytes) 2024-09-16T11:44:16.694 INFO:teuthology.orchestra.run.smithi070.stdout:4449 writesame 0x3ce4166 thru 0x3ced919 data_size 0x4c(0x97b4 bytes) 2024-09-16T11:44:16.695 INFO:teuthology.orchestra.run.smithi070.stdout:4450 read 0x1eada39 thru 0x1ebb007 (0xd5cf bytes) 2024-09-16T11:44:16.712 INFO:teuthology.orchestra.run.smithi070.stdout:4451 write 0xc7398a0 thru 0xc743313 (0x9a74 bytes) 2024-09-16T11:44:16.715 INFO:teuthology.orchestra.run.smithi070.stdout:4452 punch from 0xc29fb1d to 0xc2ad7b8, (0xdc9b bytes) 2024-09-16T11:44:16.715 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.715 INFO:teuthology.orchestra.run.smithi070.stdout:4455 write 0xa52cfd8 thru 0xa539695 (0xc6be bytes) 2024-09-16T11:44:16.725 INFO:teuthology.orchestra.run.smithi070.stdout:4459 write 0xdcc2008 thru 0xdcd1537 (0xf530 bytes) 2024-09-16T11:44:16.729 INFO:teuthology.orchestra.run.smithi070.stdout:4461 punch from 0x5b371e4 to 0x5b45d41, (0xeb5d bytes) 2024-09-16T11:44:16.729 INFO:teuthology.orchestra.run.smithi070.stdout:4462 punch from 0xc9fff6c to 0xca0870b, (0x879f bytes) 2024-09-16T11:44:16.729 INFO:teuthology.orchestra.run.smithi070.stdout:4463 read 0x3cac8cb thru 0x3cb4061 (0x7797 bytes) 2024-09-16T11:44:16.730 INFO:teuthology.orchestra.run.smithi070.stdout:4464 read 0x878464c thru 0x87887fd (0x41b2 bytes) 2024-09-16T11:44:16.730 INFO:teuthology.orchestra.run.smithi070.stdout:4465 read 0x85c7183 thru 0x85d0793 (0x9611 bytes) 2024-09-16T11:44:16.730 INFO:teuthology.orchestra.run.smithi070.stdout:4466 write 0x361442c thru 0x3620fbc (0xcb91 bytes) 2024-09-16T11:44:16.732 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.732 INFO:teuthology.orchestra.run.smithi070.stdout:4468 write 0x91c1331 thru 0x91c6551 (0x5221 bytes) 2024-09-16T11:44:16.735 INFO:teuthology.orchestra.run.smithi070.stdout:4469 read 0x5585a16 thru 0x5594950 (0xef3b bytes) 2024-09-16T11:44:16.736 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.736 INFO:teuthology.orchestra.run.smithi070.stdout:4471 write 0x3ad8b57 thru 0x3ae85d2 (0xfa7c bytes) 2024-09-16T11:44:16.738 INFO:teuthology.orchestra.run.smithi070.stdout:4472 read 0xd530f10 thru 0xd53a19d (0x928e bytes) 2024-09-16T11:44:16.752 INFO:teuthology.orchestra.run.smithi070.stdout:4473 trunc from 0xdcd1538 to 0x27273b7 2024-09-16T11:44:16.757 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:16.755+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:16.764 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.769 INFO:teuthology.orchestra.run.smithi070.stdout:4476 trunc from 0x27273b7 to 0x702146a 2024-09-16T11:44:16.775 INFO:teuthology.orchestra.run.smithi070.stdout:4477 read 0x257d9b2 thru 0x25803c6 (0x2a15 bytes) 2024-09-16T11:44:16.786 INFO:teuthology.orchestra.run.smithi070.stdout:4479 write 0xc140ce5 thru 0xc150282 (0xf59e bytes) 2024-09-16T11:44:16.789 INFO:teuthology.orchestra.run.smithi070.stdout:4480 write 0x7c9165d thru 0x7c92d95 (0x1739 bytes) 2024-09-16T11:44:16.792 INFO:teuthology.orchestra.run.smithi070.stdout:4481 read 0x337fe2c thru 0x3381ff6 (0x21cb bytes) 2024-09-16T11:44:16.792 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.792 INFO:teuthology.orchestra.run.smithi070.stdout:4483 punch from 0x7138453 to 0x714295d, (0xa50a bytes) 2024-09-16T11:44:16.792 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.806 INFO:teuthology.orchestra.run.smithi070.stdout:4485 trunc from 0xc150283 to 0x340bb6 2024-09-16T11:44:16.816 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:16.817 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:16.825 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:16.831 INFO:teuthology.orchestra.run.smithi070.stdout:4486 writesame 0x5e0f5cb thru 0x5e18cfe data_size 0x2(0x9734 bytes) 2024-09-16T11:44:16.842 INFO:teuthology.orchestra.run.smithi070.stdout:4487 write 0xadbf9d7 thru 0xadc62ff (0x6929 bytes) 2024-09-16T11:44:16.845 INFO:teuthology.orchestra.run.smithi070.stdout:4488 write 0x6584660 thru 0x65856af (0x1050 bytes) 2024-09-16T11:44:16.847 INFO:teuthology.orchestra.run.smithi070.stdout:4489 write 0x6d83f3f thru 0x6d8cca8 (0x8d6a bytes) 2024-09-16T11:44:16.850 INFO:teuthology.orchestra.run.smithi070.stdout:4490 write 0x2ab3035 thru 0x2ab7ccf (0x4c9b bytes) 2024-09-16T11:44:16.852 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.852 INFO:teuthology.orchestra.run.smithi070.stdout:4492 write 0x8d65f6c thru 0x8d6cf48 (0x6fdd bytes) 2024-09-16T11:44:16.856 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.864 INFO:teuthology.orchestra.run.smithi070.stdout:4494 write 0xe569eff thru 0xe57856b (0xe66d bytes) 2024-09-16T11:44:16.865 INFO:teuthology.orchestra.run.smithi070.stdout:4495 read 0x90ab6d1 thru 0x90b9792 (0xe0c2 bytes) 2024-09-16T11:44:16.865 INFO:teuthology.orchestra.run.smithi070.stdout:4497 read 0xa6f78ce thru 0xa6f9af6 (0x2229 bytes) 2024-09-16T11:44:16.867 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:16.865+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:16.875 INFO:teuthology.orchestra.run.smithi070.stdout:4499 trunc from 0xe57856c to 0x60d3067 2024-09-16T11:44:16.882 INFO:teuthology.orchestra.run.smithi070.stdout:4500 writesame 0x10d21e6 thru 0x10dff96 data_size 0x1d(0xddb1 bytes) 2024-09-16T11:44:16.885 INFO:teuthology.orchestra.run.smithi070.stdout:4501 write 0x3cdc3c7 thru 0x3ce520e (0x8e48 bytes) 2024-09-16T11:44:16.893 INFO:teuthology.orchestra.run.smithi070.stdout:4502 writesame 0x746f76a thru 0x7473ad4 data_size 0xb(0x436b bytes) 2024-09-16T11:44:16.899 INFO:teuthology.orchestra.run.smithi070.stdout:4504 write 0x8a4669e thru 0x8a48d2b (0x268e bytes) 2024-09-16T11:44:16.901 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.910 INFO:teuthology.orchestra.run.smithi070.stdout:4506 writesame 0xc4e88aa thru 0xc4edd72 data_size 0x5(0x54c9 bytes) 2024-09-16T11:44:16.911 INFO:teuthology.orchestra.run.smithi070.stdout:4507 write 0x5c44fa7 thru 0x5c54588 (0xf5e2 bytes) 2024-09-16T11:44:16.914 INFO:teuthology.orchestra.run.smithi070.stdout:4511 write 0x168973b thru 0x169000b (0x68d1 bytes) 2024-09-16T11:44:16.915 INFO:teuthology.orchestra.run.smithi070.stdout:4512 write 0x251596b thru 0x252053f (0xabd5 bytes) 2024-09-16T11:44:16.923 INFO:teuthology.orchestra.run.smithi070.stdout:4516 trunc from 0xc4edd73 to 0x8097877 2024-09-16T11:44:16.930 INFO:teuthology.orchestra.run.smithi070.stdout:4517 write 0x312a1b thru 0x318f11 (0x64f7 bytes) 2024-09-16T11:44:16.932 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:16.939 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:16.939 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:16.943 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:16.941+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:16.944 INFO:teuthology.orchestra.run.smithi070.stdout:4520 write 0xc7757ab thru 0xc781635 (0xbe8b bytes) 2024-09-16T11:44:16.945 INFO:teuthology.orchestra.run.smithi070.stdout:4521 write 0xc02b29f thru 0xc02fcb7 (0x4a19 bytes) 2024-09-16T11:44:16.947 INFO:teuthology.orchestra.run.smithi070.stdout:4524 write 0x80c0234 thru 0x80c3f58 (0x3d25 bytes) 2024-09-16T11:44:16.949 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:16.950 INFO:teuthology.orchestra.run.smithi070.stdout:4525 write 0xa03d6fa thru 0xa040daf (0x36b6 bytes) 2024-09-16T11:44:16.961 INFO:teuthology.orchestra.run.smithi070.stdout:4526 trunc from 0xc781636 to 0x4990306 2024-09-16T11:44:16.968 INFO:teuthology.orchestra.run.smithi070.stdout:4527 trunc from 0x4990306 to 0x4c97009 2024-09-16T11:44:16.972 INFO:teuthology.orchestra.run.smithi070.stdout:4528 punch from 0x283d813 to 0x2845fdd, (0x87ca bytes) 2024-09-16T11:44:16.989 INFO:teuthology.orchestra.run.smithi070.stdout:4529 writesame 0xe98b7c2 thru 0xe990b3b data_size 0x5(0x537a bytes) 2024-09-16T11:44:16.990 INFO:teuthology.orchestra.run.smithi070.stdout:4530 read 0x1d96dbe thru 0x1d9d2ad (0x64f0 bytes) 2024-09-16T11:44:16.991 INFO:teuthology.orchestra.run.smithi070.stdout:4531 writesame 0x52cc485 thru 0x52da8f2 data_size 0x7(0xe46e bytes) 2024-09-16T11:44:16.995 INFO:teuthology.orchestra.run.smithi070.stdout:4532 read 0x3605869 thru 0x3613c14 (0xe3ac bytes) 2024-09-16T11:44:16.996 INFO:teuthology.orchestra.run.smithi070.stdout:4533 write 0xbf16ff2 thru 0xbf2685b (0xf86a bytes) 2024-09-16T11:44:16.997 INFO:teuthology.orchestra.run.smithi070.stdout:4535 read 0x3066d01 thru 0x3067a15 (0xd15 bytes) 2024-09-16T11:44:17.005 INFO:teuthology.orchestra.run.smithi070.stdout:4537 trunc from 0xe990b3c to 0x8a41f30 2024-09-16T11:44:17.012 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:17.012 INFO:teuthology.orchestra.run.smithi070.stdout:4541 writesame 0x5df8ebb thru 0x5dfa4b1 data_size 0x1(0x15f7 bytes) 2024-09-16T11:44:17.015 INFO:teuthology.orchestra.run.smithi070.stdout:4542 read 0x5992d3e thru 0x599d09b (0xa35e bytes) 2024-09-16T11:44:17.015 INFO:teuthology.orchestra.run.smithi070.stdout:4543 read 0x64c6364 thru 0x64d00aa (0x9d47 bytes) 2024-09-16T11:44:17.025 INFO:teuthology.orchestra.run.smithi070.stdout:4544 write 0xc31949e thru 0xc31dbc8 (0x472b bytes) 2024-09-16T11:44:17.026 INFO:teuthology.orchestra.run.smithi070.stdout:4545 writesame 0xde25d4 thru 0xdee2f3 data_size 0x22(0xbd20 bytes) 2024-09-16T11:44:17.028 INFO:teuthology.orchestra.run.smithi070.stdout:4546 write 0x3da6dce thru 0x3da987b (0x2aae bytes) 2024-09-16T11:44:17.033 INFO:teuthology.orchestra.run.smithi070.stdout:4547 trunc from 0xc31dbc9 to 0x9eaa6e9 2024-09-16T11:44:17.039 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:17.040 INFO:teuthology.orchestra.run.smithi070.stdout:4550 read 0x29eb7d8 thru 0x29ee2d4 (0x2afd bytes) 2024-09-16T11:44:17.040 INFO:teuthology.orchestra.run.smithi070.stdout:4552 read 0x61dcf82 thru 0x61e0e8d (0x3f0c bytes) 2024-09-16T11:44:17.040 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:17.040 INFO:teuthology.orchestra.run.smithi070.stdout:4554 read 0x5053b12 thru 0x5057e3f (0x432e bytes) 2024-09-16T11:44:17.041 INFO:teuthology.orchestra.run.smithi070.stdout:4555 writesame 0x8b3e550 thru 0x8b4a594 data_size 0x9(0xc045 bytes) 2024-09-16T11:44:17.044 INFO:teuthology.orchestra.run.smithi070.stdout:4556 punch from 0x4118741 to 0x4126531, (0xddf0 bytes) 2024-09-16T11:44:17.044 INFO:teuthology.orchestra.run.smithi070.stdout:4557 read 0x604de1c thru 0x6052418 (0x45fd bytes) 2024-09-16T11:44:17.045 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:17.042+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:17.045 INFO:teuthology.orchestra.run.smithi070.stdout:4558 read 0x25d3b79 thru 0x25d6428 (0x28b0 bytes) 2024-09-16T11:44:17.045 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:17.050 INFO:teuthology.orchestra.run.smithi070.stdout:4560 write 0xa5a4ce4 thru 0xa5af9f9 (0xad16 bytes) 2024-09-16T11:44:17.058 INFO:teuthology.orchestra.run.smithi070.stdout:4562 trunc from 0xa5af9fa to 0xea861ed 2024-09-16T11:44:17.065 INFO:teuthology.orchestra.run.smithi070.stdout:4563 punch from 0x2d0c19f to 0x2d114c3, (0x5324 bytes) 2024-09-16T11:44:17.065 INFO:teuthology.orchestra.run.smithi070.stdout:4564 read 0x7ca44b9 thru 0x7cacf09 (0x8a51 bytes) 2024-09-16T11:44:17.065 INFO:teuthology.orchestra.run.smithi070.stdout:4565 punch from 0x75dabfc to 0x75de3e2, (0x37e6 bytes) 2024-09-16T11:44:17.065 INFO:teuthology.orchestra.run.smithi070.stdout:4566 write 0x2bb4f0f thru 0x2bc3e24 (0xef16 bytes) 2024-09-16T11:44:17.065 INFO:teuthology.orchestra.run.smithi070.stdout:4568 read 0x59d340 thru 0x5aad1a (0xd9db bytes) 2024-09-16T11:44:17.067 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:17.067 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:17.067 INFO:teuthology.orchestra.run.smithi070.stdout:4569 trunc from 0xea861ed to 0xcc11d0d 2024-09-16T11:44:17.073 INFO:teuthology.orchestra.run.smithi070.stdout:4571 writesame 0xbcdc8a2 thru 0xbce51cb data_size 0x2(0x892a bytes) 2024-09-16T11:44:17.077 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:17.090 INFO:teuthology.orchestra.run.smithi070.stdout:4572 write 0xe6232bc thru 0xe62f3d3 (0xc118 bytes) 2024-09-16T11:44:17.097 INFO:teuthology.orchestra.run.smithi070.stdout:4576 trunc from 0xe62f3d4 to 0x9488dc1 2024-09-16T11:44:17.105 INFO:teuthology.orchestra.run.smithi070.stdout:4577 write 0x74a82b1 thru 0x74ac3aa (0x40fa bytes) 2024-09-16T11:44:17.107 INFO:teuthology.orchestra.run.smithi070.stdout:4578 writesame 0x6bf8e04 thru 0x6bfe970 data_size 0x1f(0x5b6d bytes) 2024-09-16T11:44:17.109 INFO:teuthology.orchestra.run.smithi070.stdout:4579 read 0x90b8b52 thru 0x90bdf63 (0x5412 bytes) 2024-09-16T11:44:17.109 INFO:teuthology.orchestra.run.smithi070.stdout:4581 write 0x648be71 thru 0x648ef56 (0x30e6 bytes) 2024-09-16T11:44:17.110 INFO:teuthology.orchestra.run.smithi070.stdout:4582 read 0x290f235 thru 0x29182ff (0x90cb bytes) 2024-09-16T11:44:17.115 INFO:teuthology.orchestra.run.smithi070.stdout:4583 writesame 0x94d56a6 thru 0x94e1c67 data_size 0x11(0xc5c2 bytes) 2024-09-16T11:44:17.116 INFO:teuthology.orchestra.run.smithi070.stdout:4585 read 0x8aac776 thru 0x8ab8e46 (0xc6d1 bytes) 2024-09-16T11:44:17.116 INFO:teuthology.orchestra.run.smithi070.stdout:4586 read 0x69d0233 thru 0x69d7276 (0x7044 bytes) 2024-09-16T11:44:17.127 INFO:teuthology.orchestra.run.smithi070.stdout:4587 write 0xe8316ce thru 0xe8325f7 (0xf2a bytes) 2024-09-16T11:44:17.129 INFO:teuthology.orchestra.run.smithi070.stdout:4588 punch from 0x4183886 to 0x418c833, (0x8fad bytes) 2024-09-16T11:44:17.129 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:17.129 INFO:teuthology.orchestra.run.smithi070.stdout:4590 write 0x65198ae thru 0x651dbae (0x4301 bytes) 2024-09-16T11:44:17.130 INFO:teuthology.orchestra.run.smithi070.stdout:4591 write 0x2a1559b thru 0x2a1c4b9 (0x6f1f bytes) 2024-09-16T11:44:17.132 INFO:teuthology.orchestra.run.smithi070.stdout:4593 punch from 0x282145b to 0x282647f, (0x5024 bytes) 2024-09-16T11:44:17.132 INFO:teuthology.orchestra.run.smithi070.stdout:4594 read 0x8362b4d thru 0x8365922 (0x2dd6 bytes) 2024-09-16T11:44:17.132 INFO:teuthology.orchestra.run.smithi070.stdout:4596 read 0x9449a24 thru 0x944caee (0x30cb bytes) 2024-09-16T11:44:17.132 INFO:teuthology.orchestra.run.smithi070.stdout:4597 read 0x157d91f thru 0x1584b75 (0x7257 bytes) 2024-09-16T11:44:17.133 INFO:teuthology.orchestra.run.smithi070.stdout:4598 read 0x7f45b52 thru 0x7f466bb (0xb6a bytes) 2024-09-16T11:44:17.133 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:17.133 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:17.133 INFO:teuthology.orchestra.run.smithi070.stdout:4601 punch from 0xa022989 to 0xa0310dd, (0xe754 bytes) 2024-09-16T11:44:17.133 INFO:teuthology.orchestra.run.smithi070.stdout:4602 writesame 0xcab5e08 thru 0xcabc6d4 data_size 0x21(0x68cd bytes) 2024-09-16T11:44:17.134 INFO:teuthology.orchestra.run.smithi070.stdout:4603 write 0x1e2fa6e thru 0x1e3c633 (0xcbc6 bytes) 2024-09-16T11:44:17.137 INFO:teuthology.orchestra.run.smithi070.stdout:4604 writesame 0x9bf53f4 thru 0x9bfe45c data_size 0x1(0x9069 bytes) 2024-09-16T11:44:17.145 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:17.143+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:17.155 INFO:teuthology.orchestra.run.smithi070.stdout:4606 writesame 0x3c6cf6 thru 0x3c9668 data_size 0x9(0x2973 bytes) 2024-09-16T11:44:17.160 INFO:teuthology.orchestra.run.smithi070.stdout:4607 trunc from 0xe8325f8 to 0xc0a38eb 2024-09-16T11:44:17.167 INFO:teuthology.orchestra.run.smithi070.stdout:4610 write 0x3e9c0c3 thru 0x3ea768f (0xb5cd bytes) 2024-09-16T11:44:17.168 INFO:teuthology.orchestra.run.smithi070.stdout:4611 writesame 0x4edd878 thru 0x4ee6489 data_size 0x1(0x8c12 bytes) 2024-09-16T11:44:17.187 INFO:teuthology.orchestra.run.smithi070.stdout:4612 trunc from 0xc0a38eb to 0xe633e83 2024-09-16T11:44:17.193 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:17.193 INFO:teuthology.orchestra.run.smithi070.stdout:4614 write 0xabaf83 thru 0xabfef2 (0x4f70 bytes) 2024-09-16T11:44:17.194 INFO:teuthology.orchestra.run.smithi070.stdout:4615 write 0x4e624a9 thru 0x4e64432 (0x1f8a bytes) 2024-09-16T11:44:17.206 INFO:teuthology.orchestra.run.smithi070.stdout:4616 trunc from 0xe633e83 to 0x68662f5 2024-09-16T11:44:17.207 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:17.207 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:17.214 INFO:teuthology.orchestra.run.smithi070.stdout:4617 read 0x8cba9b thru 0x8db7f1 (0xfd57 bytes) 2024-09-16T11:44:17.214 INFO:teuthology.orchestra.run.smithi070.stdout:4618 read 0x6642a7f thru 0x664af60 (0x84e2 bytes) 2024-09-16T11:44:17.215 INFO:teuthology.orchestra.run.smithi070.stdout:4620 clone 29 order 21 su 524288 sc 7 2024-09-16T11:44:17.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:17.247 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:17.244+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:17.348 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:17.346+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:17.354 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:17.354 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:17.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:17.449 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:17.447+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:17.459 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:17.459 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:17.468 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:17.550 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:17.548+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:17.565 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:17.565 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:17.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:17.651 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:17.648+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:17.713 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:17.713 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:17.721 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:17.752 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:17.750+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:17.818 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:17.818 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:17.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:17.852 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:17.850+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:17.925 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:17.925 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:17.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:17.979 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:17.977+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:18.039 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:18.039 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:18.041 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:44:18.049 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:18.068 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:44:18.083 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-16T11:44:18.084 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:18.082+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:18.102 INFO:teuthology.orchestra.run.smithi070.stdout:4621 read 0x38bf3a1 thru 0x38ce135 (0xed95 bytes) 2024-09-16T11:44:18.115 INFO:teuthology.orchestra.run.smithi070.stdout:4622 write 0x89b3365 thru 0x89b90a2 (0x5d3e bytes) 2024-09-16T11:44:18.125 INFO:teuthology.orchestra.run.smithi070.stdout:4624 write 0x9c7eae4 thru 0x9c85f6e (0x748b bytes) 2024-09-16T11:44:18.136 INFO:teuthology.orchestra.run.smithi070.stdout:4625 writesame 0xac783da thru 0xac7edcb data_size 0x2(0x69f2 bytes) 2024-09-16T11:44:18.137 INFO:teuthology.orchestra.run.smithi070.stdout:4626 write 0x3b5f061 thru 0x3b5fc2c (0xbcc bytes) 2024-09-16T11:44:18.150 INFO:teuthology.orchestra.run.smithi070.stdout:4627 write 0xd2b9557 thru 0xd2c299a (0x9444 bytes) 2024-09-16T11:44:18.153 INFO:teuthology.orchestra.run.smithi070.stdout:4628 read 0x4da16a thru 0x4e7e94 (0xdd2b bytes) 2024-09-16T11:44:18.154 INFO:teuthology.orchestra.run.smithi070.stdout:4629 writesame 0xb2372dd thru 0xb23ed4d data_size 0x1(0x7a71 bytes) 2024-09-16T11:44:18.154 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:18.152+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:18.171 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:18.171 INFO:teuthology.orchestra.run.smithi070.stdout:4631 write 0x1d24bfa thru 0x1d25afa (0xf01 bytes) 2024-09-16T11:44:18.175 INFO:teuthology.orchestra.run.smithi070.stdout:4634 read 0x4ec90a6 thru 0x4ecb35a (0x22b5 bytes) 2024-09-16T11:44:18.175 INFO:teuthology.orchestra.run.smithi070.stdout:4635 writesame 0x1de5ade thru 0x1df46f1 data_size 0x3a(0xec14 bytes) 2024-09-16T11:44:18.180 INFO:teuthology.orchestra.run.smithi070.stdout:4636 read 0x2f67edc thru 0x2f6e4c7 (0x65ec bytes) 2024-09-16T11:44:18.180 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:18.180 INFO:teuthology.orchestra.run.smithi070.stdout:4639 write 0x16ca30f thru 0x16d0408 (0x60fa bytes) 2024-09-16T11:44:18.185 INFO:teuthology.orchestra.run.smithi070.stdout:4640 writesame 0xfb5868 thru 0xfbcbeb data_size 0x2(0x7384 bytes) 2024-09-16T11:44:18.194 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:18.194 INFO:teuthology.orchestra.run.smithi070.stdout:4642 write 0x212d78c thru 0x213c3c5 (0xec3a bytes) 2024-09-16T11:44:18.196 INFO:teuthology.orchestra.run.smithi070.stdout:4643 punch from 0x937bde8 to 0x9388e8a, (0xd0a2 bytes) 2024-09-16T11:44:18.196 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:18.196 INFO:teuthology.orchestra.run.smithi070.stdout:4646 read 0x15dcffd thru 0x15e8b7c (0xbb80 bytes) 2024-09-16T11:44:18.197 INFO:teuthology.orchestra.run.smithi070.stdout:4647 read 0x7f403c thru 0x7fc525 (0x84ea bytes) 2024-09-16T11:44:18.197 INFO:teuthology.orchestra.run.smithi070.stdout:4648 writesame 0xba25855 thru 0xba2990a data_size 0x21(0x40b6 bytes) 2024-09-16T11:44:18.200 INFO:teuthology.orchestra.run.smithi070.stdout:4653 read 0x5b7f154 thru 0x5b8e5a3 (0xf450 bytes) 2024-09-16T11:44:18.201 INFO:teuthology.orchestra.run.smithi070.stdout:4655 writesame 0x728c5d5 thru 0x7296e87 data_size 0x1(0xa8b3 bytes) 2024-09-16T11:44:18.202 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:18.202 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:18.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:18.221 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:18.222 INFO:teuthology.orchestra.run.smithi070.stdout:4662 trunc from 0xd2c299b to 0xdcb69ad 2024-09-16T11:44:18.232 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:18.232 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:18.242 INFO:teuthology.orchestra.run.smithi070.stdout:4663 trunc from 0xdcb69ad to 0x2474121 2024-09-16T11:44:18.252 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:18.253 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:18.255 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:18.253+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:18.266 INFO:teuthology.orchestra.run.smithi070.stdout:4664 read 0xee65a4 thru 0xeee186 (0x7be3 bytes) 2024-09-16T11:44:18.267 INFO:teuthology.orchestra.run.smithi070.stdout:4666 punch from 0x22f9a4f to 0x22feed8, (0x5489 bytes) 2024-09-16T11:44:18.267 INFO:teuthology.orchestra.run.smithi070.stdout:4668 read 0x1c9d202 thru 0x1ca783e (0xa63d bytes) 2024-09-16T11:44:18.269 INFO:teuthology.orchestra.run.smithi070.stdout:4669 trunc from 0x2474121 to 0x3ad37d5 2024-09-16T11:44:18.278 INFO:teuthology.orchestra.run.smithi070.stdout:4670 trunc from 0x3ad37d5 to 0x6942f9 2024-09-16T11:44:18.283 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:18.283 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:18.301 INFO:teuthology.orchestra.run.smithi070.stdout:4671 clone 30 order 19 su 131072 sc 10 2024-09-16T11:44:18.304 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:18.304 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:18.319 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:18.320 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:18.336 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:18.336 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:18.347 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:18.347 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:18.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:18.347 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:18.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19545 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4415488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.820249864000001, 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043429581000000002, 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19545 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:18.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:18.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:18.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:18.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523839+0000", 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0220000000000004e-06 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523845+0000", 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6500000000000001e-06 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523848+0000", 2024-09-16T11:44:18.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1879999999999998e-06 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567092+0000", 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043244145999999997 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567176+0000", 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3825999999999993e-05 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567263+0000", 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7423999999999995e-05 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19546 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4734976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.820232023999999, 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043599231000000002, 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.352 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.353 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.353 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19546 2024-09-16T11:44:18.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.353 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:18.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:18.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523954+0000", 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010217099999999999 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523957+0000", 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3270000000000001e-06 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523966+0000", 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6489999999999997e-06 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523971+0000", 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567291+0000", 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043319931999999998 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567360+0000", 2024-09-16T11:44:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8628999999999996e-05 2024-09-16T11:44:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567451+0000", 2024-09-16T11:44:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1012000000000002e-05 2024-09-16T11:44:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19547 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4841472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.357 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:18.357 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.820107167, 2024-09-16T11:44:18.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043658858000000002, 2024-09-16T11:44:18.357 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.357 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.357 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.357 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.357 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.357 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19547 2024-09-16T11:44:18.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.357 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525102+0000", 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011257730000000001 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525106+0000", 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9910000000000004e-06 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525116+0000", 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0231000000000001e-05 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525137+0000", 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0828000000000001e-05 2024-09-16T11:44:18.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567478+0000", 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.042340467999999999 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567554+0000", 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6074000000000004e-05 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567635+0000", 2024-09-16T11:44:18.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1493000000000001e-05 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19553 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4947968~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.818880105000002, 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043450397000000002, 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.361 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.362 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19553 2024-09-16T11:44:18.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.362 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:18.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:18.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1799999999999999e-07 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525205+0000", 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.062e-06 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525208+0000", 2024-09-16T11:44:18.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8739999999999999e-06 2024-09-16T11:44:18.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525209+0000", 2024-09-16T11:44:18.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.725e-06 2024-09-16T11:44:18.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568520+0000", 2024-09-16T11:44:18.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043310226 2024-09-16T11:44:18.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568569+0000", 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8959999999999999e-05 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568654+0000", 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5332000000000002e-05 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19554 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5267456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.365 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.818871113, 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043659587, 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19554 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:18.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:18.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527103+0000", 2024-09-16T11:44:18.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018907419999999999 2024-09-16T11:44:18.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527108+0000", 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6040000000000003e-06 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527118+0000", 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527122+0000", 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9559999999999996e-06 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568688+0000", 2024-09-16T11:44:18.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041565116999999999 2024-09-16T11:44:18.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568768+0000", 2024-09-16T11:44:18.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0474000000000002e-05 2024-09-16T11:44:18.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568872+0000", 2024-09-16T11:44:18.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104028 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19565 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6332416~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.816792541000002, 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043494890000000001, 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.370 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.371 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19565 2024-09-16T11:44:18.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.371 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:18.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:18.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527292+0000", 2024-09-16T11:44:18.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.27e-07 2024-09-16T11:44:18.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527293+0000", 2024-09-16T11:44:18.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.917e-06 2024-09-16T11:44:18.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527299+0000", 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6699999999999999e-06 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527304+0000", 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6160000000000004e-06 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570683+0000", 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043379171000000001 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.374 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.374 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570733+0000", 2024-09-16T11:44:18.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0485000000000002e-05 2024-09-16T11:44:18.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.374 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.374 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570786+0000", 2024-09-16T11:44:18.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2604e-05 2024-09-16T11:44:18.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.374 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.375 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19566 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6758400~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.375 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:18.375 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.816776207, 2024-09-16T11:44:18.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043643183000000002, 2024-09-16T11:44:18.375 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.375 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.375 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19566 2024-09-16T11:44:18.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.375 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:18.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:18.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527308+0000", 2024-09-16T11:44:18.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:44:18.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527310+0000", 2024-09-16T11:44:18.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.68e-06 2024-09-16T11:44:18.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527316+0000", 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9730000000000002e-06 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527319+0000", 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.027e-06 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570810+0000", 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043491525000000003 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570882+0000", 2024-09-16T11:44:18.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:44:18.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570951+0000", 2024-09-16T11:44:18.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8782000000000007e-05 2024-09-16T11:44:18.379 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.379 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.379 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.380 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19567 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.380 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:18.380 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.816761500999998, 2024-09-16T11:44:18.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043786291999999997, 2024-09-16T11:44:18.380 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.380 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.380 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.380 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.380 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.380 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19567 2024-09-16T11:44:18.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.380 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:18.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:18.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529108+0000", 2024-09-16T11:44:18.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001785656 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529113+0000", 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1080000000000001e-06 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529122+0000", 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0550000000000005e-06 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529127+0000", 2024-09-16T11:44:18.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9849999999999997e-06 2024-09-16T11:44:18.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570997+0000", 2024-09-16T11:44:18.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041869811 2024-09-16T11:44:18.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571063+0000", 2024-09-16T11:44:18.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6340999999999994e-05 2024-09-16T11:44:18.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571108+0000", 2024-09-16T11:44:18.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5336000000000001e-05 2024-09-16T11:44:18.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.384 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.384 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19580 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 8036352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.384 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:18.384 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.8147862, 2024-09-16T11:44:18.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043451596000000002, 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19580 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531106+0000", 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001808179 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531110+0000", 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5990000000000004e-06 2024-09-16T11:44:18.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531123+0000", 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2687000000000001e-05 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531129+0000", 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572642+0000", 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041513474000000002 2024-09-16T11:44:18.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572691+0000", 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8989000000000001e-05 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572749+0000", 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7806999999999997e-05 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.389 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19708 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4530176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.389 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:18.389 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.812739635, 2024-09-16T11:44:18.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394863999999998, 2024-09-16T11:44:18.389 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.389 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.389 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.389 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.389 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.389 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19708 2024-09-16T11:44:18.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.389 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.67e-07 2024-09-16T11:44:18.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531346+0000", 2024-09-16T11:44:18.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.477e-06 2024-09-16T11:44:18.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531351+0000", 2024-09-16T11:44:18.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1050000000000003e-06 2024-09-16T11:44:18.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531354+0000", 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9960000000000002e-06 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574620+0000", 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043266365000000001 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574662+0000", 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1662000000000002e-05 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574739+0000", 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6991999999999997e-05 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19709 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4849664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.812726159, 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043537291999999998, 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.393 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.394 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19709 2024-09-16T11:44:18.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.394 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:18.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531358+0000", 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531360+0000", 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531365+0000", 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.716e-06 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531369+0000", 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5410000000000001e-06 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574763+0000", 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394359 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574831+0000", 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7997999999999995e-05 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574895+0000", 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3427000000000001e-05 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.397 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19710 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4956160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.812710880000001, 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043674198999999997, 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19710 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533105+0000", 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001731961 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533109+0000", 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3080000000000001e-06 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533120+0000", 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0625000000000001e-05 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533125+0000", 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574923+0000", 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041797187 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574991+0000", 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8752999999999997e-05 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.575047+0000", 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5596000000000001e-05 2024-09-16T11:44:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19719 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5488640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.810739130999998, 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043412013999999999, 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19719 2024-09-16T11:44:18.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533345+0000", 2024-09-16T11:44:18.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533347+0000", 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8139999999999999e-06 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533352+0000", 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4970000000000001e-06 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533355+0000", 2024-09-16T11:44:18.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.711e-06 2024-09-16T11:44:18.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576629+0000", 2024-09-16T11:44:18.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043274295999999997 2024-09-16T11:44:18.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576706+0000", 2024-09-16T11:44:18.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6711000000000005e-05 2024-09-16T11:44:18.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576757+0000", 2024-09-16T11:44:18.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0509e-05 2024-09-16T11:44:18.406 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.406 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.406 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.406 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19720 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5808128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.406 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:18.406 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.810726482, 2024-09-16T11:44:18.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043577006000000001, 2024-09-16T11:44:18.406 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19720 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:18.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:18.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.35e-07 2024-09-16T11:44:18.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533359+0000", 2024-09-16T11:44:18.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:44:18.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533364+0000", 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4349999999999996e-06 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533367+0000", 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2349999999999999e-06 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576786+0000", 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043418885999999997 2024-09-16T11:44:18.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576862+0000", 2024-09-16T11:44:18.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6172000000000002e-05 2024-09-16T11:44:18.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576934+0000", 2024-09-16T11:44:18.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1891999999999995e-05 2024-09-16T11:44:18.410 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.410 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.410 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19721 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6234112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.810712962, 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043681656999999999, 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19721 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-07 2024-09-16T11:44:18.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533373+0000", 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.973e-06 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533377+0000", 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.403e-06 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533379+0000", 2024-09-16T11:44:18.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.058e-06 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576959+0000", 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043579433000000001 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577015+0000", 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5859000000000002e-05 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577052+0000", 2024-09-16T11:44:18.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7620999999999998e-05 2024-09-16T11:44:18.415 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.415 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.415 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.415 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19722 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6340608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.415 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:18.415 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.810702235000001, 2024-09-16T11:44:18.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043854922999999997, 2024-09-16T11:44:18.415 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.415 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.416 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19722 2024-09-16T11:44:18.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.416 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:18.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:18.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535103+0000", 2024-09-16T11:44:18.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017217020000000001 2024-09-16T11:44:18.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535107+0000", 2024-09-16T11:44:18.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2819999999999998e-06 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535118+0000", 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0148e-05 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535123+0000", 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577091+0000", 2024-09-16T11:44:18.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041967930000000001 2024-09-16T11:44:18.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577163+0000", 2024-09-16T11:44:18.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1853999999999993e-05 2024-09-16T11:44:18.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577236+0000", 2024-09-16T11:44:18.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3699e-05 2024-09-16T11:44:18.419 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.419 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.420 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.420 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19732 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6447104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.420 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:18.420 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.808842379000001, 2024-09-16T11:44:18.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043399048000000003, 2024-09-16T11:44:18.420 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.420 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.420 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.420 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.420 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.421 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19732 2024-09-16T11:44:18.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.421 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:18.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:18.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535242+0000", 2024-09-16T11:44:18.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:18.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535243+0000", 2024-09-16T11:44:18.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.793e-06 2024-09-16T11:44:18.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535249+0000", 2024-09-16T11:44:18.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.643e-06 2024-09-16T11:44:18.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535252+0000", 2024-09-16T11:44:18.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:44:18.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578501+0000", 2024-09-16T11:44:18.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043248343000000002 2024-09-16T11:44:18.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578576+0000", 2024-09-16T11:44:18.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5544999999999996e-05 2024-09-16T11:44:18.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578640+0000", 2024-09-16T11:44:18.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4018999999999998e-05 2024-09-16T11:44:18.424 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.424 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.424 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.425 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19733 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6873088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.425 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:18.425 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.808821538, 2024-09-16T11:44:18.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043549681, 2024-09-16T11:44:18.425 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.425 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.425 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.425 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.425 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.425 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19733 2024-09-16T11:44:18.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.425 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535263+0000", 2024-09-16T11:44:18.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:44:18.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535265+0000", 2024-09-16T11:44:18.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9369999999999998e-06 2024-09-16T11:44:18.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535271+0000", 2024-09-16T11:44:18.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9719999999999997e-06 2024-09-16T11:44:18.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535274+0000", 2024-09-16T11:44:18.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:44:18.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578668+0000", 2024-09-16T11:44:18.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043393355000000002 2024-09-16T11:44:18.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578744+0000", 2024-09-16T11:44:18.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6853000000000004e-05 2024-09-16T11:44:18.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578812+0000", 2024-09-16T11:44:18.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.737e-05 2024-09-16T11:44:18.429 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.429 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.429 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19734 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7299072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.808805791000001, 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043699189999999999, 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19734 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537101+0000", 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001823098 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537105+0000", 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537119+0000", 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614999999999999e-05 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537125+0000", 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:44:18.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578838+0000", 2024-09-16T11:44:18.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041712605999999999 2024-09-16T11:44:18.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578912+0000", 2024-09-16T11:44:18.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4396999999999999e-05 2024-09-16T11:44:18.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578977+0000", 2024-09-16T11:44:18.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4858999999999997e-05 2024-09-16T11:44:18.434 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.434 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.434 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.434 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19851 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4431872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.434 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:18.434 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.806725047, 2024-09-16T11:44:18.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043564540999999998, 2024-09-16T11:44:18.435 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:18.435 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:18.435 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:18.435 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:18.435 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.435 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19851 2024-09-16T11:44:18.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.435 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:18.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:18.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:18.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539100+0000", 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017412079999999999 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:18.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539105+0000", 2024-09-16T11:44:18.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T11:44:18.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:18.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539116+0000", 2024-09-16T11:44:18.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1824999999999999e-05 2024-09-16T11:44:18.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539122+0000", 2024-09-16T11:44:18.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9859999999999999e-06 2024-09-16T11:44:18.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580764+0000", 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041641299 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580839+0000", 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5167999999999996e-05 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580923+0000", 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4314999999999995e-05 2024-09-16T11:44:18.438 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.439 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.439 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.439 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:18.439 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:18.439 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:18.440 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:18.440 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:18.441 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:18.441 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:18.442 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:18.442 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:18.438+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:18.540 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:18.540 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:18.541 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:18.541 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:18.538+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:18.549 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:18.595 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:44:18.595 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:44:18.595 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-16T11:44:18.595 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:44:18.596 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:44:18.596 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-16T11:44:18.642 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:18.640+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:18.686 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:18.686 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:18.686 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:18.687 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:18.687 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:18.695 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:18.695 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:18.695 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "" 2024-09-16T11:44:18.696 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:18.744 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:18.742+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:18.795 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.431199703000001, 2024-09-16T11:44:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:44:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:44:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:44:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:44:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:44:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:44:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:44:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:44:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:44:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.431177674000001, 2024-09-16T11:44:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:44:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:44:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:44:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:44:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:44:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:44:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:44:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:44:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:44:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.431162909000001, 2024-09-16T11:44:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:44:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:44:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:44:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:44:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:44:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:44:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:44:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:44:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:44:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.431142841, 2024-09-16T11:44:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:44:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:44:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:44:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:44:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:44:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:44:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:44:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:44:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:44:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:44:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:44:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.431112004999999, 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:44:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:44:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:44:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:44:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:44:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:44:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:44:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:44:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:44:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:44:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:44:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:44:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:44:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.431092485000001, 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:44:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:44:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:44:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:44:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.431064906, 2024-09-16T11:44:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:44:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:44:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:44:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:44:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:44:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.431046211000002, 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:44:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:44:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:44:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:44:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:44:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.431031202, 2024-09-16T11:44:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:44:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:44:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:44:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:44:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:44:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:44:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:44:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:44:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:44:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:44:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:44:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:44:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:44:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:44:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.430999104000001, 2024-09-16T11:44:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:44:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:44:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:44:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:44:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:44:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:44:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:44:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:44:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:44:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:44:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:44:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:44:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:44:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:44:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:44:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:44:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.430975485000001, 2024-09-16T11:44:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:44:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:44:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:44:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:44:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:44:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.430960756000001, 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:44:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:44:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:44:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:44:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:44:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:44:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:44:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:44:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:44:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:44:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:44:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:44:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:44:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.430938932, 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:44:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:44:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:44:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:44:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:44:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:44:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:44:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.430924281999999, 2024-09-16T11:44:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:44:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:44:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:44:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.430902709000001, 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:44:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.430883262999998, 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:44:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:44:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:44:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:44:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:44:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:44:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:44:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:44:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:44:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.430846566, 2024-09-16T11:44:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:44:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:44:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:44:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:44:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:44:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:44:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:44:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:44:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:44:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.430824619999999, 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:44:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:44:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:44:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:44:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:44:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:44:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.430802734, 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:44:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:44:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:44:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:44:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:44:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:44:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.430788015000001, 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:44:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:44:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:44:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:44:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:44:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:44:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:44:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:18.888 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:18.888 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:18.886+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:18.952 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T11:44:18.992 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:18.992 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:19.001 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:19.006 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone29 from 0x6942f9 (overlap 0x6942f9) to 0x0 2024-09-16T11:44:19.007 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:19.005+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:19.049 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-16T11:44:19.090 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:19.087+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:19.102 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:19.102 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:19.102 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:19.111 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:19.187 INFO:teuthology.orchestra.run.smithi070.stdout:4672 write 0xe867ed5 thru 0xe86f408 (0x7534 bytes) 2024-09-16T11:44:19.190 INFO:teuthology.orchestra.run.smithi070.stdout:4673 punch from 0xcd5a45d to 0xcd5aa15, (0x5b8 bytes) 2024-09-16T11:44:19.190 INFO:teuthology.orchestra.run.smithi070.stdout:4674 punch from 0x72b8fd8 to 0x72c72c2, (0xe2ea bytes) 2024-09-16T11:44:19.191 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:19.188+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:19.195 INFO:teuthology.orchestra.run.smithi070.stdout:4675 trunc from 0xe86f409 to 0xaee94a2 2024-09-16T11:44:19.203 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:19.203 INFO:teuthology.orchestra.run.smithi070.stdout:4677 read 0x6f81d86 thru 0x6f8ba98 (0x9d13 bytes) 2024-09-16T11:44:19.214 INFO:teuthology.orchestra.run.smithi070.stdout:4678 trunc from 0xaee94a2 to 0x1c8e334 2024-09-16T11:44:19.220 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:19.220 INFO:teuthology.orchestra.run.smithi070.stdout:4680 punch from 0x136c166 to 0x1374761, (0x85fb bytes) 2024-09-16T11:44:19.220 INFO:teuthology.orchestra.run.smithi070.stdout:4683 punch from 0x19651e6 to 0x196c478, (0x7292 bytes) 2024-09-16T11:44:19.234 INFO:teuthology.orchestra.run.smithi070.stdout:4684 trunc from 0x1c8e334 to 0xdb7a9dd 2024-09-16T11:44:19.239 INFO:teuthology.orchestra.run.smithi070.stdout:4685 read 0x5a39766 thru 0x5a3cf03 (0x379e bytes) 2024-09-16T11:44:19.239 INFO:teuthology.orchestra.run.smithi070.stdout:4686 read 0xceb5eca thru 0xcebc0a7 (0x61de bytes) 2024-09-16T11:44:19.239 INFO:teuthology.orchestra.run.smithi070.stdout:4690 write 0xc00bfb1 thru 0xc015401 (0x9451 bytes) 2024-09-16T11:44:19.243 INFO:teuthology.orchestra.run.smithi070.stdout:4692 punch from 0x9510870 to 0x951cb7f, (0xc30f bytes) 2024-09-16T11:44:19.243 INFO:teuthology.orchestra.run.smithi070.stdout:4693 write 0x87aa9ba thru 0x87b663d (0xbc84 bytes) 2024-09-16T11:44:19.253 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:19.253 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:19.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:19.253 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:19.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.253 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19942 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1990656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.253 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:19.253 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723998890000001, 2024-09-16T11:44:19.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027866354999999999, 2024-09-16T11:44:19.253 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.253 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.254 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19942 2024-09-16T11:44:19.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.254 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:19.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526279+0000", 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4899999999999995e-07 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526281+0000", 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526287+0000", 2024-09-16T11:44:19.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2199999999999997e-06 2024-09-16T11:44:19.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526291+0000", 2024-09-16T11:44:19.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:44:19.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554068+0000", 2024-09-16T11:44:19.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027777076000000001 2024-09-16T11:44:19.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554104+0000", 2024-09-16T11:44:19.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5537000000000002e-05 2024-09-16T11:44:19.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554145+0000", 2024-09-16T11:44:19.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1350000000000002e-05 2024-09-16T11:44:19.257 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.257 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.257 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19943 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2416640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723982586999998, 2024-09-16T11:44:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027947257999999999, 2024-09-16T11:44:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19943 2024-09-16T11:44:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5999999999999999e-07 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526297+0000", 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8059999999999999e-06 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526303+0000", 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6579999999999999e-06 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526306+0000", 2024-09-16T11:44:19.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3969999999999998e-06 2024-09-16T11:44:19.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554165+0000", 2024-09-16T11:44:19.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027858519000000002 2024-09-16T11:44:19.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554201+0000", 2024-09-16T11:44:19.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5908000000000002e-05 2024-09-16T11:44:19.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554242+0000", 2024-09-16T11:44:19.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1510000000000001e-05 2024-09-16T11:44:19.262 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.262 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.262 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.262 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19944 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2842624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.262 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:19.262 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723967691999999, 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028052639000000001, 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19944 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526312+0000", 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:44:19.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526317+0000", 2024-09-16T11:44:19.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5910000000000003e-06 2024-09-16T11:44:19.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526321+0000", 2024-09-16T11:44:19.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2339999999999999e-06 2024-09-16T11:44:19.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554263+0000", 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027942181 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554307+0000", 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4499999999999997e-05 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554362+0000", 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5078999999999999e-05 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.267 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19945 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3162112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.267 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:19.267 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723953136999999, 2024-09-16T11:44:19.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028145673999999999, 2024-09-16T11:44:19.267 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.267 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.267 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19945 2024-09-16T11:44:19.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526325+0000", 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526326+0000", 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.733e-06 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526332+0000", 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526336+0000", 2024-09-16T11:44:19.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8410000000000002e-06 2024-09-16T11:44:19.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554389+0000", 2024-09-16T11:44:19.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028053576 2024-09-16T11:44:19.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554427+0000", 2024-09-16T11:44:19.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7299999999999999e-05 2024-09-16T11:44:19.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554470+0000", 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3195000000000002e-05 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19946 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3268608~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723937557999999, 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028232587, 2024-09-16T11:44:19.271 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19946 2024-09-16T11:44:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526346+0000", 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6459999999999998e-06 2024-09-16T11:44:19.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526352+0000", 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1070000000000001e-06 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526355+0000", 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554493+0000", 2024-09-16T11:44:19.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.02813746 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554530+0000", 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6881999999999997e-05 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554572+0000", 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2688999999999997e-05 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19947 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4014080~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723918204, 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028315646999999999, 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19947 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526360+0000", 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.01e-07 2024-09-16T11:44:19.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526361+0000", 2024-09-16T11:44:19.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8300000000000001e-06 2024-09-16T11:44:19.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526369+0000", 2024-09-16T11:44:19.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:44:19.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526377+0000", 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0909999999999993e-06 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554593+0000", 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028216657999999999 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554632+0000", 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8828000000000002e-05 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554675+0000", 2024-09-16T11:44:19.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2710000000000003e-05 2024-09-16T11:44:19.280 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.280 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.280 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.280 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19948 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3588096~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.280 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:19.280 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723895805000002, 2024-09-16T11:44:19.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028394545, 2024-09-16T11:44:19.281 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.281 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.281 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.281 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.281 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.281 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19948 2024-09-16T11:44:19.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.281 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:19.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9200000000000001e-07 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526384+0000", 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526391+0000", 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5150000000000003e-06 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526394+0000", 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4240000000000002e-06 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554695+0000", 2024-09-16T11:44:19.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028301218999999999 2024-09-16T11:44:19.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554731+0000", 2024-09-16T11:44:19.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6298999999999999e-05 2024-09-16T11:44:19.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554776+0000", 2024-09-16T11:44:19.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4675999999999997e-05 2024-09-16T11:44:19.284 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.285 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.285 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.285 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19949 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1351680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.285 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:19.285 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723879533000002, 2024-09-16T11:44:19.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028490081, 2024-09-16T11:44:19.285 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.285 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.285 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.285 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.285 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19949 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:19.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:44:19.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526400+0000", 2024-09-16T11:44:19.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7099999999999999e-06 2024-09-16T11:44:19.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526406+0000", 2024-09-16T11:44:19.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5140000000000001e-06 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526409+0000", 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5360000000000001e-06 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554801+0000", 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028392378999999999 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554843+0000", 2024-09-16T11:44:19.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1056000000000001e-05 2024-09-16T11:44:19.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554888+0000", 2024-09-16T11:44:19.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466999999999997e-05 2024-09-16T11:44:19.289 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.289 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.289 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.289 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19950 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1777664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.289 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723864537000001, 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028661844999999998, 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19950 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:19.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526415+0000", 2024-09-16T11:44:19.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.776e-06 2024-09-16T11:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526422+0000", 2024-09-16T11:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6930000000000001e-06 2024-09-16T11:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526425+0000", 2024-09-16T11:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554923+0000", 2024-09-16T11:44:19.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028498102000000001 2024-09-16T11:44:19.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555000+0000", 2024-09-16T11:44:19.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6619999999999995e-05 2024-09-16T11:44:19.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555075+0000", 2024-09-16T11:44:19.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4931999999999993e-05 2024-09-16T11:44:19.293 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19951 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2097152~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723848726, 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028755718, 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.294 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19951 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:19.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-16T11:44:19.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526431+0000", 2024-09-16T11:44:19.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:44:19.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526441+0000", 2024-09-16T11:44:19.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7790000000000007e-06 2024-09-16T11:44:19.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526445+0000", 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.634e-06 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555100+0000", 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654722000000001 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555138+0000", 2024-09-16T11:44:19.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7996e-05 2024-09-16T11:44:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555184+0000", 2024-09-16T11:44:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6427999999999997e-05 2024-09-16T11:44:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19952 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2203648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723828166000001, 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028901157, 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19952 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:19.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526450+0000", 2024-09-16T11:44:19.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:44:19.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526452+0000", 2024-09-16T11:44:19.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9039999999999999e-06 2024-09-16T11:44:19.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526465+0000", 2024-09-16T11:44:19.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3559e-05 2024-09-16T11:44:19.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526470+0000", 2024-09-16T11:44:19.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-09-16T11:44:19.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555217+0000", 2024-09-16T11:44:19.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028747287999999999 2024-09-16T11:44:19.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555272+0000", 2024-09-16T11:44:19.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4885000000000001e-05 2024-09-16T11:44:19.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555350+0000", 2024-09-16T11:44:19.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8428000000000002e-05 2024-09-16T11:44:19.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.303 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.303 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19953 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2523136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.303 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:19.303 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723803574000002, 2024-09-16T11:44:19.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029004309999999998, 2024-09-16T11:44:19.303 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.303 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.304 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19953 2024-09-16T11:44:19.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.304 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:19.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:19.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:19.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-16T11:44:19.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526476+0000", 2024-09-16T11:44:19.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9810000000000002e-06 2024-09-16T11:44:19.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526482+0000", 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526485+0000", 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2760000000000001e-06 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555383+0000", 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028898113 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555433+0000", 2024-09-16T11:44:19.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9753999999999998e-05 2024-09-16T11:44:19.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555478+0000", 2024-09-16T11:44:19.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5006999999999998e-05 2024-09-16T11:44:19.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.307 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.308 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19954 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2949120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.308 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:19.308 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723788577000001, 2024-09-16T11:44:19.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029165054999999999, 2024-09-16T11:44:19.308 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.308 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.308 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19954 2024-09-16T11:44:19.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.308 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:19.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:19.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:19.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1799999999999995e-07 2024-09-16T11:44:19.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526491+0000", 2024-09-16T11:44:19.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7320000000000001e-06 2024-09-16T11:44:19.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526497+0000", 2024-09-16T11:44:19.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6869999999999999e-06 2024-09-16T11:44:19.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526500+0000", 2024-09-16T11:44:19.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:44:19.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555513+0000", 2024-09-16T11:44:19.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029012626 2024-09-16T11:44:19.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555576+0000", 2024-09-16T11:44:19.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3288999999999997e-05 2024-09-16T11:44:19.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555654+0000", 2024-09-16T11:44:19.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7861999999999994e-05 2024-09-16T11:44:19.315 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.315 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.315 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.315 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19955 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3375104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.315 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:19.315 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723769274999999, 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029241979000000001, 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19955 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:19.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526509+0000", 2024-09-16T11:44:19.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-09-16T11:44:19.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526510+0000", 2024-09-16T11:44:19.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.844e-06 2024-09-16T11:44:19.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526516+0000", 2024-09-16T11:44:19.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9039999999999997e-06 2024-09-16T11:44:19.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526520+0000", 2024-09-16T11:44:19.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3639999999999999e-06 2024-09-16T11:44:19.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555669+0000", 2024-09-16T11:44:19.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029149628 2024-09-16T11:44:19.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555706+0000", 2024-09-16T11:44:19.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7017000000000002e-05 2024-09-16T11:44:19.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555750+0000", 2024-09-16T11:44:19.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3856999999999997e-05 2024-09-16T11:44:19.320 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.320 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.320 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.320 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19956 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3694592~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.320 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:19.320 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723754320000001, 2024-09-16T11:44:19.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029398799999999999, 2024-09-16T11:44:19.320 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.320 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19956 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526524+0000", 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526525+0000", 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526531+0000", 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6010000000000002e-06 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526535+0000", 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.597e-06 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555785+0000", 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029250235999999999 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555845+0000", 2024-09-16T11:44:19.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0393000000000003e-05 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555922+0000", 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6710000000000002e-05 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19957 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4120576~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.723739167000002, 2024-09-16T11:44:19.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029481157000000001, 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19957 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526539+0000", 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526540+0000", 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.759e-06 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526546+0000", 2024-09-16T11:44:19.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1079999999999998e-06 2024-09-16T11:44:19.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526550+0000", 2024-09-16T11:44:19.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.269e-06 2024-09-16T11:44:19.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555939+0000", 2024-09-16T11:44:19.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029389444000000001 2024-09-16T11:44:19.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555977+0000", 2024-09-16T11:44:19.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7487999999999998e-05 2024-09-16T11:44:19.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.556019+0000", 2024-09-16T11:44:19.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2756999999999997e-05 2024-09-16T11:44:19.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.328 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:28959 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.240882755000001, 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031145285000000002, 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 28959 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009396+0000", 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.296e-06 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030864+0000", 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.021468451 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030892+0000", 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7555999999999998e-05 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030902+0000", 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9550000000000004e-06 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030979+0000", 2024-09-16T11:44:19.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7137000000000004e-05 2024-09-16T11:44:19.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031051+0000", 2024-09-16T11:44:19.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1475000000000003e-05 2024-09-16T11:44:19.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:19.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031389+0000", 2024-09-16T11:44:19.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033827100000000001 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.039264+0000", 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078751700000000008 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040484+0000", 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012199579999999999 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:19.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040517+0000", 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2960999999999998e-05 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040540+0000", 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3054999999999999e-05 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30605 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 3915776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.006576496000001, 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938655999999999, 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30605 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243704+0000", 2024-09-16T11:44:19.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2199999999999999e-06 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243710+0000", 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6340000000000003e-06 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243714+0000", 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5030000000000002e-06 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271450+0000", 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027736035999999999 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271536+0000", 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6521999999999996e-05 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271640+0000", 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103341 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.338 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30607 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4046848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.006553341, 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028130446, 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30607 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.339 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:19.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:19.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243725+0000", 2024-09-16T11:44:19.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2399999999999998e-07 2024-09-16T11:44:19.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243727+0000", 2024-09-16T11:44:19.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3379999999999999e-06 2024-09-16T11:44:19.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243733+0000", 2024-09-16T11:44:19.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4280000000000001e-06 2024-09-16T11:44:19.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243737+0000", 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8990000000000003e-06 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271676+0000", 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938481000000001 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271757+0000", 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1396999999999993e-05 2024-09-16T11:44:19.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271855+0000", 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7379000000000004e-05 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30608 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.006536489999998, 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028323879, 2024-09-16T11:44:19.343 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.344 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.344 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30608 2024-09-16T11:44:19.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.344 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:19.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243743+0000", 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.762e-06 2024-09-16T11:44:19.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243749+0000", 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5670000000000002e-06 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243758+0000", 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8309999999999996e-06 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271879+0000", 2024-09-16T11:44:19.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028120501999999999 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271965+0000", 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5755999999999997e-05 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.272065+0000", 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000100101 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.348 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.348 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:19.349 INFO:teuthology.orchestra.run.smithi070.stdout:4694 trunc from 0xdb7a9dd to 0x4468261 2024-09-16T11:44:19.350 INFO:teuthology.orchestra.run.smithi070.stdout:4697 read 0x2916903 thru 0x2918456 (0x1b54 bytes) 2024-09-16T11:44:19.350 INFO:teuthology.orchestra.run.smithi070.stdout:4698 read 0x7aa1dd thru 0x7abf68 (0x1d8c bytes) 2024-09-16T11:44:19.350 INFO:teuthology.orchestra.run.smithi070.stdout:4699 writesame 0x5f38f53 thru 0x5f3dda3 data_size 0x29(0x4e51 bytes) 2024-09-16T11:44:19.350 INFO:teuthology.orchestra.run.smithi070.stdout:4700 write 0xc5ccc8e thru 0xc5d2621 (0x5994 bytes) 2024-09-16T11:44:19.350 INFO:teuthology.orchestra.run.smithi070.stdout:4701 clone 31 order 18 su 32768 sc 2 2024-09-16T11:44:19.350 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:19.351 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:19.347+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:19.449 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:19.448+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:19.489 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:19.489 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:19.489 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:19.489 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:19.497 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:19.550 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:19.548+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:19.595 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:19.596 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:19.604 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:19.652 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:19.650+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:19.745 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:19.745 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:19.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:19.746 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:19.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.746 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19282 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.746 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:19.746 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.269763174000001, 2024-09-16T11:44:19.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521151000000002, 2024-09-16T11:44:19.746 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.746 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.746 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.746 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.746 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.746 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19282 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472838+0000", 2024-09-16T11:44:19.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:44:19.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472848+0000", 2024-09-16T11:44:19.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9799999999999993e-06 2024-09-16T11:44:19.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472858+0000", 2024-09-16T11:44:19.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9429999999999995e-06 2024-09-16T11:44:19.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472862+0000", 2024-09-16T11:44:19.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4460000000000003e-06 2024-09-16T11:44:19.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501277+0000", 2024-09-16T11:44:19.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028414900999999999 2024-09-16T11:44:19.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501322+0000", 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4880999999999999e-05 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501358+0000", 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6245000000000003e-05 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19286 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7979008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:19.750 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.269390242, 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521860999999999, 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19286 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:19.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473211+0000", 2024-09-16T11:44:19.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:44:19.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473214+0000", 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1080000000000001e-06 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473226+0000", 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2441e-05 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473234+0000", 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729999999999998e-06 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501592+0000", 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028358491999999999 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501671+0000", 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8846999999999999e-05 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501732+0000", 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1118000000000007e-05 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.754 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20619 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7610368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.182041074000001, 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028500152000000001, 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20619 2024-09-16T11:44:19.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.756 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:19.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:19.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560560+0000", 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9499999999999998e-07 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560561+0000", 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.071e-06 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560565+0000", 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560567+0000", 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588949+0000", 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381878999999999 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588989+0000", 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0649000000000003e-05 2024-09-16T11:44:19.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589059+0000", 2024-09-16T11:44:19.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0239000000000003e-05 2024-09-16T11:44:19.759 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.759 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.759 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.759 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20620 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7716864~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:19.759 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:19.759 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.182031674000001, 2024-09-16T11:44:19.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028627621999999998, 2024-09-16T11:44:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20620 2024-09-16T11:44:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:19.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562377+0000", 2024-09-16T11:44:19.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018081729999999999 2024-09-16T11:44:19.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562380+0000", 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0469999999999998e-06 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562386+0000", 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2040000000000002e-06 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562390+0000", 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589092+0000", 2024-09-16T11:44:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026701798999999998 2024-09-16T11:44:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589139+0000", 2024-09-16T11:44:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7320000000000001e-05 2024-09-16T11:44:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589196+0000", 2024-09-16T11:44:19.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7201999999999999e-05 2024-09-16T11:44:19.764 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.764 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.764 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.764 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41954 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.764 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:19.764 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328325884, 2024-09-16T11:44:19.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028594441000000002, 2024-09-16T11:44:19.764 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.764 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41954 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414275+0000", 2024-09-16T11:44:19.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:44:19.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414277+0000", 2024-09-16T11:44:19.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:19.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414282+0000", 2024-09-16T11:44:19.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7559999999999996e-06 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414286+0000", 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442778+0000", 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028492237 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442816+0000", 2024-09-16T11:44:19.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8198000000000003e-05 2024-09-16T11:44:19.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442869+0000", 2024-09-16T11:44:19.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2589000000000001e-05 2024-09-16T11:44:19.769 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.769 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.769 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.769 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41955 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.769 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:19.769 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328310825999999, 2024-09-16T11:44:19.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028687865, 2024-09-16T11:44:19.769 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.769 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.769 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.769 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.770 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.770 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41955 2024-09-16T11:44:19.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.770 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:19.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:19.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414292+0000", 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414299+0000", 2024-09-16T11:44:19.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4279999999999999e-06 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414303+0000", 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442892+0000", 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028589564000000001 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442930+0000", 2024-09-16T11:44:19.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330999999999999e-05 2024-09-16T11:44:19.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442977+0000", 2024-09-16T11:44:19.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7874000000000003e-05 2024-09-16T11:44:19.773 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.773 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.773 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.773 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41956 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328275509000001, 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028766896, 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41956 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414326+0000", 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9900000000000009e-07 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414330+0000", 2024-09-16T11:44:19.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2339999999999996e-06 2024-09-16T11:44:19.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414341+0000", 2024-09-16T11:44:19.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1358999999999999e-05 2024-09-16T11:44:19.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414347+0000", 2024-09-16T11:44:19.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8100000000000003e-06 2024-09-16T11:44:19.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443002+0000", 2024-09-16T11:44:19.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654331000000002 2024-09-16T11:44:19.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443040+0000", 2024-09-16T11:44:19.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8262000000000001e-05 2024-09-16T11:44:19.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443092+0000", 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1901000000000003e-05 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41957 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328248090000001, 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028848318000000001, 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.779 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.779 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41957 2024-09-16T11:44:19.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.779 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:19.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:19.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414353+0000", 2024-09-16T11:44:19.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:19.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414355+0000", 2024-09-16T11:44:19.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3489999999999999e-06 2024-09-16T11:44:19.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414364+0000", 2024-09-16T11:44:19.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1449999999999997e-06 2024-09-16T11:44:19.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414368+0000", 2024-09-16T11:44:19.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8940000000000003e-06 2024-09-16T11:44:19.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443115+0000", 2024-09-16T11:44:19.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028746532000000002 2024-09-16T11:44:19.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443152+0000", 2024-09-16T11:44:19.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6814999999999999e-05 2024-09-16T11:44:19.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443201+0000", 2024-09-16T11:44:19.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9069000000000001e-05 2024-09-16T11:44:19.783 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.783 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.783 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.783 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41958 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.784 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:19.784 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328224307999999, 2024-09-16T11:44:19.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028939856, 2024-09-16T11:44:19.784 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.784 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.784 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.784 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.784 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.784 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41958 2024-09-16T11:44:19.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.784 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:19.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:19.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414377+0000", 2024-09-16T11:44:19.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4099999999999999e-07 2024-09-16T11:44:19.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414379+0000", 2024-09-16T11:44:19.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.418e-06 2024-09-16T11:44:19.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414385+0000", 2024-09-16T11:44:19.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999999e-06 2024-09-16T11:44:19.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414389+0000", 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7740000000000002e-06 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443223+0000", 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028833919 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443263+0000", 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0148999999999998e-05 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443316+0000", 2024-09-16T11:44:19.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2636000000000002e-05 2024-09-16T11:44:19.788 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.788 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.788 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.788 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41959 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.788 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:19.788 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328207383000001, 2024-09-16T11:44:19.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029040309, 2024-09-16T11:44:19.788 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41959 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:19.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:19.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:44:19.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414395+0000", 2024-09-16T11:44:19.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7579999999999999e-06 2024-09-16T11:44:19.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414401+0000", 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5330000000000002e-06 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414404+0000", 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.529e-06 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443339+0000", 2024-09-16T11:44:19.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028934933999999999 2024-09-16T11:44:19.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443383+0000", 2024-09-16T11:44:19.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3832000000000003e-05 2024-09-16T11:44:19.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443433+0000", 2024-09-16T11:44:19.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0309000000000002e-05 2024-09-16T11:44:19.792 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.792 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.792 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41960 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328192549000001, 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029138134, 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41960 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.793 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:19.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:19.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:19.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-07 2024-09-16T11:44:19.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414410+0000", 2024-09-16T11:44:19.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:44:19.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414417+0000", 2024-09-16T11:44:19.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:19.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414420+0000", 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443457+0000", 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029037065000000001 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443495+0000", 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7555000000000002e-05 2024-09-16T11:44:19.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443546+0000", 2024-09-16T11:44:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1422000000000003e-05 2024-09-16T11:44:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41961 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328176568, 2024-09-16T11:44:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029230973, 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41961 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4499999999999998e-07 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414432+0000", 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.413e-06 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414439+0000", 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4769999999999997e-06 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414443+0000", 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8859999999999997e-06 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443569+0000", 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029126124 2024-09-16T11:44:19.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443606+0000", 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6588999999999998e-05 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443655+0000", 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9138999999999998e-05 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41962 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328153601, 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029324505000000001, 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41962 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:19.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:19.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:19.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:44:19.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414449+0000", 2024-09-16T11:44:19.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0279999999999999e-06 2024-09-16T11:44:19.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414456+0000", 2024-09-16T11:44:19.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3339999999999998e-06 2024-09-16T11:44:19.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414459+0000", 2024-09-16T11:44:19.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.382e-06 2024-09-16T11:44:19.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443678+0000", 2024-09-16T11:44:19.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029218958999999999 2024-09-16T11:44:19.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443719+0000", 2024-09-16T11:44:19.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0722000000000001e-05 2024-09-16T11:44:19.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443771+0000", 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2661000000000003e-05 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41963 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328137698000001, 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029416557999999999, 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.806 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41963 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414465+0000", 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414470+0000", 2024-09-16T11:44:19.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4750000000000001e-06 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414474+0000", 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3720000000000001e-06 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443795+0000", 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029320821 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443830+0000", 2024-09-16T11:44:19.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5463000000000002e-05 2024-09-16T11:44:19.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443879+0000", 2024-09-16T11:44:19.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9147000000000002e-05 2024-09-16T11:44:19.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.810 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.810 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41964 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.810 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:19.811 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.32812281, 2024-09-16T11:44:19.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029521202999999999, 2024-09-16T11:44:19.811 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.811 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.811 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41964 2024-09-16T11:44:19.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.811 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414480+0000", 2024-09-16T11:44:19.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:44:19.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414486+0000", 2024-09-16T11:44:19.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:19.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414489+0000", 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2770000000000001e-06 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443903+0000", 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029414109000000001 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443944+0000", 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0992999999999999e-05 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443999+0000", 2024-09-16T11:44:19.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4632000000000001e-05 2024-09-16T11:44:19.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.815 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.815 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41965 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.815 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:19.815 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328107848, 2024-09-16T11:44:19.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029611293, 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41965 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:19.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:19.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:19.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414495+0000", 2024-09-16T11:44:19.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.88e-06 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414501+0000", 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2489999999999998e-06 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414504+0000", 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3139999999999999e-06 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444023+0000", 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029518564000000001 2024-09-16T11:44:19.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444058+0000", 2024-09-16T11:44:19.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5196e-05 2024-09-16T11:44:19.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444104+0000", 2024-09-16T11:44:19.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5732999999999997e-05 2024-09-16T11:44:19.819 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.819 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.819 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.820 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41966 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.820 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:19.820 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328091642, 2024-09-16T11:44:19.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029707678000000001, 2024-09-16T11:44:19.820 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.820 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.820 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41966 2024-09-16T11:44:19.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.820 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:19.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:19.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:19.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:19.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414514+0000", 2024-09-16T11:44:19.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-16T11:44:19.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414518+0000", 2024-09-16T11:44:19.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3839999999999999e-06 2024-09-16T11:44:19.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414521+0000", 2024-09-16T11:44:19.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0400000000000001e-06 2024-09-16T11:44:19.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444128+0000", 2024-09-16T11:44:19.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029606612000000001 2024-09-16T11:44:19.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444167+0000", 2024-09-16T11:44:19.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9418000000000001e-05 2024-09-16T11:44:19.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444216+0000", 2024-09-16T11:44:19.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9354999999999998e-05 2024-09-16T11:44:19.824 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.824 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.824 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.824 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41967 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.824 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:19.825 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328075833, 2024-09-16T11:44:19.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029796916, 2024-09-16T11:44:19.825 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.825 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.825 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41967 2024-09-16T11:44:19.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.825 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6699999999999999e-07 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414527+0000", 2024-09-16T11:44:19.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7060000000000001e-06 2024-09-16T11:44:19.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414533+0000", 2024-09-16T11:44:19.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2140000000000001e-06 2024-09-16T11:44:19.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414536+0000", 2024-09-16T11:44:19.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:19.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444239+0000", 2024-09-16T11:44:19.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029702698999999999 2024-09-16T11:44:19.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444273+0000", 2024-09-16T11:44:19.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4229e-05 2024-09-16T11:44:19.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444321+0000", 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8477999999999999e-05 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41968 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328060936, 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029902765000000001, 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.829 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.830 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41968 2024-09-16T11:44:19.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.830 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:19.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:19.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414540+0000", 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414542+0000", 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414547+0000", 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4639999999999997e-06 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414551+0000", 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6279999999999998e-06 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444346+0000", 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029794885 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444389+0000", 2024-09-16T11:44:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3442000000000002e-05 2024-09-16T11:44:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444442+0000", 2024-09-16T11:44:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3173000000000001e-05 2024-09-16T11:44:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41969 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.328046024000001, 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029997612999999999, 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41969 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414555+0000", 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414556+0000", 2024-09-16T11:44:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.725e-06 2024-09-16T11:44:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414563+0000", 2024-09-16T11:44:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2530000000000001e-06 2024-09-16T11:44:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414566+0000", 2024-09-16T11:44:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4429999999999999e-06 2024-09-16T11:44:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444463+0000", 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029897271 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444503+0000", 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9782e-05 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444552+0000", 2024-09-16T11:44:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8789000000000003e-05 2024-09-16T11:44:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:19.838 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:19.839 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:19.840 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:19.837+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:19.940 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:19.938+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:19.981 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:19.981 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:19.981 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:19.981 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:19.990 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:20.015 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone30 from 0xc5d2622 (overlap 0x6942f9) to 0xa86b2 2024-09-16T11:44:20.074 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:20.072+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:20.078 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-16T11:44:20.079 INFO:teuthology.orchestra.run.smithi070.stdout:4702 writesame 0xbb96228 thru 0xbb9f877 data_size 0x68(0x9650 bytes) 2024-09-16T11:44:20.086 INFO:teuthology.orchestra.run.smithi070.stdout:4704 punch from 0x1b605ee to 0x1b6a1a4, (0x9bb6 bytes) 2024-09-16T11:44:20.086 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.086 INFO:teuthology.orchestra.run.smithi070.stdout:4706 write 0xbd5aadf thru 0xbd60137 (0x5659 bytes) 2024-09-16T11:44:20.090 INFO:teuthology.orchestra.run.smithi070.stdout:4707 trunc from 0xc5d2622 to 0xb4f6cb6 2024-09-16T11:44:20.105 INFO:teuthology.orchestra.run.smithi070.stdout:4708 punch from 0x68d438 to 0x68f4d6, (0x209e bytes) 2024-09-16T11:44:20.105 INFO:teuthology.orchestra.run.smithi070.stdout:4711 writesame 0x2c268f1 thru 0x2c2d784 data_size 0x2a(0x6e94 bytes) 2024-09-16T11:44:20.111 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.111 INFO:teuthology.orchestra.run.smithi070.stdout:4714 read 0x9d64e71 thru 0x9d6c934 (0x7ac4 bytes) 2024-09-16T11:44:20.111 INFO:teuthology.orchestra.run.smithi070.stdout:4716 punch from 0xae180e7 to 0xae20e0f, (0x8d28 bytes) 2024-09-16T11:44:20.112 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.112 INFO:teuthology.orchestra.run.smithi070.stdout:4718 punch from 0x99e2129 to 0x99e6854, (0x472b bytes) 2024-09-16T11:44:20.112 INFO:teuthology.orchestra.run.smithi070.stdout:4719 read 0x751a420 thru 0x7525a5c (0xb63d bytes) 2024-09-16T11:44:20.112 INFO:teuthology.orchestra.run.smithi070.stdout:4722 write 0x6e4e5de thru 0x6e4fa56 (0x1479 bytes) 2024-09-16T11:44:20.113 INFO:teuthology.orchestra.run.smithi070.stdout:4724 read 0x3f115c thru 0x3f5285 (0x412a bytes) 2024-09-16T11:44:20.114 INFO:teuthology.orchestra.run.smithi070.stdout:4725 write 0x1d50735 thru 0x1d5cf00 (0xc7cc bytes) 2024-09-16T11:44:20.119 INFO:teuthology.orchestra.run.smithi070.stdout:4726 writesame 0x5440d79 thru 0x54428c9 data_size 0x25(0x1b51 bytes) 2024-09-16T11:44:20.125 INFO:teuthology.orchestra.run.smithi070.stdout:4728 trunc from 0xb4f6cb6 to 0xe2d5859 2024-09-16T11:44:20.130 INFO:teuthology.orchestra.run.smithi070.stdout:4732 read 0xdb99937 thru 0xdba5b82 (0xc24c bytes) 2024-09-16T11:44:20.130 INFO:teuthology.orchestra.run.smithi070.stdout:4733 punch from 0x7904b74 to 0x7905085, (0x511 bytes) 2024-09-16T11:44:20.130 INFO:teuthology.orchestra.run.smithi070.stdout:4737 writesame 0x8a0c977 thru 0x8a12650 data_size 0x5(0x5cda bytes) 2024-09-16T11:44:20.137 INFO:teuthology.orchestra.run.smithi070.stdout:4738 writesame 0x3423b8f thru 0x342a3a3 data_size 0x5(0x6815 bytes) 2024-09-16T11:44:20.138 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:20.139 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:20.142 INFO:teuthology.orchestra.run.smithi070.stdout:4739 punch from 0x3165080 to 0x3172b57, (0xdad7 bytes) 2024-09-16T11:44:20.142 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.144 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:20.140+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:20.145 INFO:teuthology.orchestra.run.smithi070.stdout:4742 write 0x625b939 thru 0x62687ed (0xceb5 bytes) 2024-09-16T11:44:20.148 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:20.149 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.149 INFO:teuthology.orchestra.run.smithi070.stdout:4744 read 0x368dbd9 thru 0x3699d6a (0xc192 bytes) 2024-09-16T11:44:20.149 INFO:teuthology.orchestra.run.smithi070.stdout:4745 punch from 0xd5b3f11 to 0xd5b5156, (0x1245 bytes) 2024-09-16T11:44:20.149 INFO:teuthology.orchestra.run.smithi070.stdout:4747 read 0xdf67b50 thru 0xdf6a92f (0x2de0 bytes) 2024-09-16T11:44:20.149 INFO:teuthology.orchestra.run.smithi070.stdout:4748 write 0xa6199b4 thru 0xa61c673 (0x2cc0 bytes) 2024-09-16T11:44:20.153 INFO:teuthology.orchestra.run.smithi070.stdout:4749 punch from 0x1182b74 to 0x118ca5c, (0x9ee8 bytes) 2024-09-16T11:44:20.153 INFO:teuthology.orchestra.run.smithi070.stdout:4751 read 0xc9213d2 thru 0xc927f60 (0x6b8f bytes) 2024-09-16T11:44:20.153 INFO:teuthology.orchestra.run.smithi070.stdout:4752 read 0x8910240 thru 0x8917d3c (0x7afd bytes) 2024-09-16T11:44:20.153 INFO:teuthology.orchestra.run.smithi070.stdout:4753 read 0x25e7c8d thru 0x25ea6ee (0x2a62 bytes) 2024-09-16T11:44:20.153 INFO:teuthology.orchestra.run.smithi070.stdout:4754 read 0xa141894 thru 0xa14483e (0x2fab bytes) 2024-09-16T11:44:20.153 INFO:teuthology.orchestra.run.smithi070.stdout:4755 write 0xc30fdd2 thru 0xc319900 (0x9b2f bytes) 2024-09-16T11:44:20.158 INFO:teuthology.orchestra.run.smithi070.stdout:4756 read 0x63b7275 thru 0x63be1af (0x6f3b bytes) 2024-09-16T11:44:20.163 INFO:teuthology.orchestra.run.smithi070.stdout:4757 writesame 0xea85abe thru 0xea8fef9 data_size 0x2e(0xa43c bytes) 2024-09-16T11:44:20.163 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.166 INFO:teuthology.orchestra.run.smithi070.stdout:4759 trunc from 0xea8fefa to 0xba8cf2b 2024-09-16T11:44:20.176 INFO:teuthology.orchestra.run.smithi070.stdout:4760 writesame 0xba04149 thru 0xba0c58d data_size 0x1(0x8445 bytes) 2024-09-16T11:44:20.188 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.188 INFO:teuthology.orchestra.run.smithi070.stdout:4762 read 0x1d60f79 thru 0x1d6a53b (0x95c3 bytes) 2024-09-16T11:44:20.188 INFO:teuthology.orchestra.run.smithi070.stdout:4763 read 0x76b742f thru 0x76bfc8d (0x885f bytes) 2024-09-16T11:44:20.195 INFO:teuthology.orchestra.run.smithi070.stdout:4764 write 0xcfe5663 thru 0xcff2cfb (0xd699 bytes) 2024-09-16T11:44:20.198 INFO:teuthology.orchestra.run.smithi070.stdout:4765 writesame 0x54c50a8 thru 0x54cf88b data_size 0x23(0xa7e4 bytes) 2024-09-16T11:44:20.209 INFO:teuthology.orchestra.run.smithi070.stdout:4767 writesame 0xe882779 thru 0xe88af04 data_size 0x32(0x878c bytes) 2024-09-16T11:44:20.209 INFO:teuthology.orchestra.run.smithi070.stdout:4768 read 0x9c394dd thru 0x9c3d6d6 (0x41fa bytes) 2024-09-16T11:44:20.210 INFO:teuthology.orchestra.run.smithi070.stdout:4769 punch from 0x803043a to 0x8031a40, (0x1606 bytes) 2024-09-16T11:44:20.210 INFO:teuthology.orchestra.run.smithi070.stdout:4770 write 0x4780b7 thru 0x47924c (0x1196 bytes) 2024-09-16T11:44:20.213 INFO:teuthology.orchestra.run.smithi070.stdout:4772 flatten 2024-09-16T11:44:20.243 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:20.240+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:20.245 INFO:teuthology.orchestra.run.smithi070.stdout:4773 writesame 0xe8ab2d thru 0xe9281f data_size 0x1(0x7cf3 bytes) 2024-09-16T11:44:20.258 INFO:teuthology.orchestra.run.smithi070.stdout:4774 write 0xa2e4f81 thru 0xa2e505b (0xdb bytes) 2024-09-16T11:44:20.261 INFO:teuthology.orchestra.run.smithi070.stdout:4775 writesame 0xd7c2826 thru 0xd7cc1a5 data_size 0x40(0x9980 bytes) 2024-09-16T11:44:20.275 INFO:teuthology.orchestra.run.smithi070.stdout:4778 trunc from 0xe88af05 to 0x603ddf7 2024-09-16T11:44:20.299 INFO:teuthology.orchestra.run.smithi070.stdout:4779 writesame 0xbf30626 thru 0xbf31afc data_size 0xb(0x14d7 bytes) 2024-09-16T11:44:20.299 INFO:teuthology.orchestra.run.smithi070.stdout:4780 read 0x5a4d36 thru 0x5b24b5 (0xd780 bytes) 2024-09-16T11:44:20.300 INFO:teuthology.orchestra.run.smithi070.stdout:4781 writesame 0x92fc7b3 thru 0x930723f data_size 0x1(0xaa8d bytes) 2024-09-16T11:44:20.301 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:20.301 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:20.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:20.301 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:20.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.301 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:20.302 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:20.302 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.378390604000003, 2024-09-16T11:44:20.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:44:20.302 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.302 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.302 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:20.302 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:20.302 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:20.302 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:44:20.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.302 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:20.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:20.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:20.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:44:20.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:44:20.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:44:20.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:44:20.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:44:20.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:44:20.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:44:20.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:44:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:44:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:44:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:44:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:44:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:44:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.308 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:20.308 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:20.308 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.378344912000003, 2024-09-16T11:44:20.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:44:20.308 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.308 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:20.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:20.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:20.308 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:44:20.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.308 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:44:20.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:44:20.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:44:20.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:44:20.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:44:20.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:44:20.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:44:20.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:44:20.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:44:20.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:44:20.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:44:20.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:44:20.313 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.313 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.313 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.378287982000003, 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:20.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:20.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:44:20.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:20.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:44:20.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:44:20.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:44:20.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:44:20.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:44:20.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:44:20.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:44:20.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:44:20.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:44:20.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:44:20.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:44:20.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.320 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:20.320 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:20.320 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.378262613999993, 2024-09-16T11:44:20.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:44:20.320 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.320 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:20.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:20.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:20.320 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:44:20.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.320 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:20.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:44:20.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:44:20.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:44:20.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:44:20.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:44:20.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:44:20.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.378210645999999, 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:44:20.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:44:20.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:44:20.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:44:20.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:44:20.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:44:20.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:44:20.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:44:20.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:44:20.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:44:20.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.331 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.378129536000003, 2024-09-16T11:44:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:44:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:44:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:44:20.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:44:20.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:44:20.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:44:20.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:44:20.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:44:20.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:20.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:44:20.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:44:20.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:20.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:44:20.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:44:20.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:20.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:44:20.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:44:20.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:44:20.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:44:20.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:44:20.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:44:20.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.337 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.378103558000007, 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:44:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:44:20.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:44:20.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:44:20.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:20.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:44:20.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:20.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:44:20.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:44:20.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:44:20.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:44:20.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:20.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:44:20.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:44:20.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:20.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:44:20.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:44:20.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:20.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.344 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:20.344 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:20.344 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 66.960584933000007, 2024-09-16T11:44:20.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:44:20.344 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.344 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:20.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:20.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:20.344 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:44:20.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.345 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:20.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:20.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:44:20.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:44:20.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:20.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:44:20.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:44:20.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:20.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:44:20.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:44:20.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:20.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:44:20.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:44:20.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:44:20.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:44:20.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:44:20.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:44:20.349 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.350 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.350 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.350 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:20.350 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:20.350 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 66.960563002000001, 2024-09-16T11:44:20.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:44:20.350 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.350 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.350 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:20.350 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:20.350 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:20.351 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:44:20.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.351 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:20.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:20.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:44:20.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:20.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:44:20.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:44:20.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:44:20.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:44:20.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:20.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:44:20.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:44:20.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:20.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:44:20.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:44:20.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:20.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:44:20.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:44:20.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:44:20.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:44:20.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:44:20.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:44:20.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.357 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.357 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:20.357 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:20.357 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 66.960486114000005, 2024-09-16T11:44:20.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:44:20.357 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.357 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:20.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:20.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:44:20.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:44:20.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:44:20.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:44:20.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:44:20.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:44:20.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:44:20.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:44:20.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:44:20.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:44:20.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:20.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:44:20.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:44:20.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:44:20.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:44:20.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:44:20.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28187 4.17 e52/28 4:ee916bd9:::rbd_data.11c1ce740b46.0000000000000000:head v 52'195, mlcod=52'193)", 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.789066728000002, 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075854429999999999, 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:20.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:20.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509349+0000", 2024-09-16T11:44:20.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2300000000000001e-06 2024-09-16T11:44:20.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513315+0000", 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039660590000000001 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513328+0000", 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.308e-05 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513334+0000", 2024-09-16T11:44:20.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513385+0000", 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1347000000000002e-05 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513407+0000", 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1852999999999999e-05 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516882+0000", 2024-09-16T11:44:20.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034749690000000001 2024-09-16T11:44:20.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516933+0000", 2024-09-16T11:44:20.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0952999999999998e-05 2024-09-16T11:44:20.367 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.367 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.367 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.367 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28959 4.8 e53/28 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head v 53'408, mlcod=53'406)", 2024-09-16T11:44:20.367 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.266917672000002, 2024-09-16T11:44:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088740420000000004, 2024-09-16T11:44:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031498+0000", 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.652e-06 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033515+0000", 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020165930000000001 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033539+0000", 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4295999999999999e-05 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033548+0000", 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1239999999999994e-06 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033610+0000", 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1371999999999994e-05 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033637+0000", 2024-09-16T11:44:20.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7098999999999999e-05 2024-09-16T11:44:20.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040341+0000", 2024-09-16T11:44:20.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067045819999999997 2024-09-16T11:44:20.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040371+0000", 2024-09-16T11:44:20.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9323999999999999e-05 2024-09-16T11:44:20.372 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.372 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.372 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.372 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32303 4.8 e53/28 4:1436ac82:::rbd_data.11c1cab081b3.0000000000000022:head v 53'417, mlcod=53'415)", 2024-09-16T11:44:20.372 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:20.372 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.796491698000001, 2024-09-16T11:44:20.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085565160000000001, 2024-09-16T11:44:20.372 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.372 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.372 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:20.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:20.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:20.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-09-16T11:44:20.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501930+0000", 2024-09-16T11:44:20.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7649999999999996e-06 2024-09-16T11:44:20.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501946+0000", 2024-09-16T11:44:20.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5301000000000001e-05 2024-09-16T11:44:20.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501952+0000", 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502100+0000", 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014847600000000001 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502120+0000", 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9303000000000001e-05 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510436+0000", 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083166420000000008 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510479+0000", 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2979000000000003e-05 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.376 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32306 4.1e e53/28 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head v 53'124, mlcod=53'122)", 2024-09-16T11:44:20.377 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:20.377 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.796425704000001, 2024-09-16T11:44:20.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084990900000000008, 2024-09-16T11:44:20.377 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.377 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.377 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:20.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:20.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:20.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:44:20.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501994+0000", 2024-09-16T11:44:20.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:44:20.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502002+0000", 2024-09-16T11:44:20.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3839999999999997e-06 2024-09-16T11:44:20.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502007+0000", 2024-09-16T11:44:20.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6290000000000001e-06 2024-09-16T11:44:20.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502454+0000", 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00044736800000000002 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502471+0000", 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6648999999999999e-05 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510455+0000", 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0079841909999999999 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510488+0000", 2024-09-16T11:44:20.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2948999999999999e-05 2024-09-16T11:44:20.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.381 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.381 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32307 4.8 e53/28 4:11c1c37d:::rbd_data.11c1cab081b3.0000000000000026:head v 53'418, mlcod=53'415)", 2024-09-16T11:44:20.381 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:20.381 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.795670369, 2024-09-16T11:44:20.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00776212, 2024-09-16T11:44:20.381 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:20.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:20.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502750+0000", 2024-09-16T11:44:20.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:44:20.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502760+0000", 2024-09-16T11:44:20.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:44:20.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502767+0000", 2024-09-16T11:44:20.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4219999999999997e-06 2024-09-16T11:44:20.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502802+0000", 2024-09-16T11:44:20.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5089999999999998e-05 2024-09-16T11:44:20.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502816+0000", 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4092e-05 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510483+0000", 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00766704 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510506+0000", 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3195999999999999e-05 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.385 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.386 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.386 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33583 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'416, mlcod=53'411)", 2024-09-16T11:44:20.386 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:20.386 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.08771196, 2024-09-16T11:44:20.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078132459999999994, 2024-09-16T11:44:20.386 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.386 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.386 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210703+0000", 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210706+0000", 2024-09-16T11:44:20.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-06 2024-09-16T11:44:20.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210712+0000", 2024-09-16T11:44:20.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:44:20.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210715+0000", 2024-09-16T11:44:20.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:44:20.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210736+0000", 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0293999999999999e-05 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210746+0000", 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0178e-05 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218477+0000", 2024-09-16T11:44:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007731501 2024-09-16T11:44:20.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218516+0000", 2024-09-16T11:44:20.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8148000000000001e-05 2024-09-16T11:44:20.390 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.390 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.390 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.390 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33584 4.0 e53/28 4:044e4d16:::rbd_data.11c1cab081b3.0000000000000019:head v 53'176, mlcod=53'174)", 2024-09-16T11:44:20.390 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:20.390 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.086900477, 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070059470000000002, 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:20.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.37e-07 2024-09-16T11:44:20.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211524+0000", 2024-09-16T11:44:20.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4830000000000005e-06 2024-09-16T11:44:20.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211535+0000", 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0842000000000001e-05 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211539+0000", 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9740000000000004e-06 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211584+0000", 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5655000000000002e-05 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211602+0000", 2024-09-16T11:44:20.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7756000000000001e-05 2024-09-16T11:44:20.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218497+0000", 2024-09-16T11:44:20.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0068944590000000004 2024-09-16T11:44:20.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218520+0000", 2024-09-16T11:44:20.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3141000000000001e-05 2024-09-16T11:44:20.394 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33590 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'419, mlcod=53'415)", 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.076653819000001, 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082499879999999998, 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:20.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:20.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221761+0000", 2024-09-16T11:44:20.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-16T11:44:20.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221765+0000", 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1590000000000003e-06 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221774+0000", 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7490000000000004e-06 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221778+0000", 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0419999999999996e-06 2024-09-16T11:44:20.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221800+0000", 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1926999999999999e-05 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221810+0000", 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0079e-05 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.229980+0000", 2024-09-16T11:44:20.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081698469999999992 2024-09-16T11:44:20.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230010+0000", 2024-09-16T11:44:20.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0746e-05 2024-09-16T11:44:20.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.399 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.399 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33591 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'420, mlcod=53'415)", 2024-09-16T11:44:20.399 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:20.399 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.076092334999998, 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007702576, 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222326+0000", 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222337+0000", 2024-09-16T11:44:20.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1008e-05 2024-09-16T11:44:20.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222342+0000", 2024-09-16T11:44:20.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0050000000000004e-06 2024-09-16T11:44:20.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222366+0000", 2024-09-16T11:44:20.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3696000000000001e-05 2024-09-16T11:44:20.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222385+0000", 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230012+0000", 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076270940000000001 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230025+0000", 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2112e-05 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.403 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.404 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.404 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33593 4.1e e53/28 4:7e77cec2:::rbd_data.11c1cab081b3.000000000000000c:head v 53'133, mlcod=53'131)", 2024-09-16T11:44:20.404 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:20.404 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.071458412999998, 2024-09-16T11:44:20.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070516270000000004, 2024-09-16T11:44:20.404 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:20.404 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:20.404 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:20.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:20.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226957+0000", 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:20.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227014+0000", 2024-09-16T11:44:20.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7074999999999998e-05 2024-09-16T11:44:20.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:20.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227024+0000", 2024-09-16T11:44:20.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0617e-05 2024-09-16T11:44:20.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:20.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227030+0000", 2024-09-16T11:44:20.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3279999999999996e-06 2024-09-16T11:44:20.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227061+0000", 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1718999999999999e-05 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227074+0000", 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2738000000000001e-05 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.233975+0000", 2024-09-16T11:44:20.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0069008480000000002 2024-09-16T11:44:20.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:20.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:20.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:20.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.234008+0000", 2024-09-16T11:44:20.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2614000000000003e-05 2024-09-16T11:44:20.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.408 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:20.408 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:20.408 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:20.411 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:20.412 INFO:teuthology.orchestra.run.smithi070.stdout:4782 read 0xb3b082e thru 0xb3b48f9 (0x40cc bytes) 2024-09-16T11:44:20.412 INFO:teuthology.orchestra.run.smithi070.stdout:4783 read 0x98ebdd1 thru 0x98f0e3f (0x506f bytes) 2024-09-16T11:44:20.412 INFO:teuthology.orchestra.run.smithi070.stdout:4784 punch from 0x1c58dbc to 0x1c5f36e, (0x65b2 bytes) 2024-09-16T11:44:20.412 INFO:teuthology.orchestra.run.smithi070.stdout:4785 trunc from 0xbf31afd to 0xc636947 2024-09-16T11:44:20.412 INFO:teuthology.orchestra.run.smithi070.stdout:4787 trunc from 0xc636947 to 0x291106a 2024-09-16T11:44:20.412 INFO:teuthology.orchestra.run.smithi070.stdout:4788 punch from 0xc86846 to 0xc8f463, (0x8c1d bytes) 2024-09-16T11:44:20.412 INFO:teuthology.orchestra.run.smithi070.stdout:4791 write 0x3b2e751 thru 0x3b3a576 (0xbe26 bytes) 2024-09-16T11:44:20.412 INFO:teuthology.orchestra.run.smithi070.stdout:4793 write 0x75c71f2 thru 0x75c8e30 (0x1c3f bytes) 2024-09-16T11:44:20.412 INFO:teuthology.orchestra.run.smithi070.stdout:4794 write 0x5049499 thru 0x504eeb5 (0x5a1d bytes) 2024-09-16T11:44:20.413 INFO:teuthology.orchestra.run.smithi070.stdout:4795 read 0xb56cf6 thru 0xb65029 (0xe334 bytes) 2024-09-16T11:44:20.413 INFO:teuthology.orchestra.run.smithi070.stdout:4796 trunc from 0x75c8e31 to 0xd560b8d 2024-09-16T11:44:20.413 INFO:teuthology.orchestra.run.smithi070.stdout:4797 write 0x34a1f36 thru 0x34a7a11 (0x5adc bytes) 2024-09-16T11:44:20.413 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.413 INFO:teuthology.orchestra.run.smithi070.stdout:4801 writesame 0x5498b86 thru 0x54a444c data_size 0x1(0xb8c7 bytes) 2024-09-16T11:44:20.413 INFO:teuthology.orchestra.run.smithi070.stdout:4802 punch from 0x6b3d180 to 0x6b3fc36, (0x2ab6 bytes) 2024-09-16T11:44:20.413 INFO:teuthology.orchestra.run.smithi070.stdout:4804 read 0x1ef8533 thru 0x1eff17d (0x6c4b bytes) 2024-09-16T11:44:20.413 INFO:teuthology.orchestra.run.smithi070.stdout:4805 writesame 0xc05e91c thru 0xc06be8a data_size 0xd(0xd56f bytes) 2024-09-16T11:44:20.413 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:20.407+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:20.414 INFO:teuthology.orchestra.run.smithi070.stdout:4806 read 0x5811156 thru 0x5813a17 (0x28c2 bytes) 2024-09-16T11:44:20.414 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.414 INFO:teuthology.orchestra.run.smithi070.stdout:4808 punch from 0x9e74012 to 0x9e780aa, (0x4098 bytes) 2024-09-16T11:44:20.414 INFO:teuthology.orchestra.run.smithi070.stdout:4809 read 0x537f6c5 thru 0x538d5f1 (0xdf2d bytes) 2024-09-16T11:44:20.415 INFO:teuthology.orchestra.run.smithi070.stdout:4810 trunc from 0xd560b8d to 0xde7b76a 2024-09-16T11:44:20.417 INFO:teuthology.orchestra.run.smithi070.stdout:4811 punch from 0xbec703f to 0xbece1d8, (0x7199 bytes) 2024-09-16T11:44:20.418 INFO:teuthology.orchestra.run.smithi070.stdout:4812 write 0x49c80e6 thru 0x49d1bb1 (0x9acc bytes) 2024-09-16T11:44:20.421 INFO:teuthology.orchestra.run.smithi070.stdout:4813 writesame 0x15ca5f thru 0x16a70a data_size 0x3a(0xdcac bytes) 2024-09-16T11:44:20.423 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.424 INFO:teuthology.orchestra.run.smithi070.stdout:4815 read 0x9aa96a4 thru 0x9aae2c3 (0x4c20 bytes) 2024-09-16T11:44:20.424 INFO:teuthology.orchestra.run.smithi070.stdout:4817 read 0xeb5c81 thru 0xeb883e (0x2bbe bytes) 2024-09-16T11:44:20.424 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.434 INFO:teuthology.orchestra.run.smithi070.stdout:4820 trunc from 0xde7b76a to 0x53a4f3e 2024-09-16T11:44:20.443 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:44:20.445 INFO:teuthology.orchestra.run.smithi070.stdout:4822 trunc from 0x53a4f3e to 0x4dad0af 2024-09-16T11:44:20.470 INFO:teuthology.orchestra.run.smithi070.stdout:4823 writesame 0xc94b37f thru 0xc95b20b data_size 0x1(0xfe8d bytes) 2024-09-16T11:44:20.470 INFO:teuthology.orchestra.run.smithi070.stdout:4824 read 0xc09efc thru 0xc10edc (0x6fe1 bytes) 2024-09-16T11:44:20.475 INFO:teuthology.orchestra.run.smithi070.stdout:4825 writesame 0xcff7d39 thru 0xd007ae8 data_size 0x7b(0xfdb0 bytes) 2024-09-16T11:44:20.475 INFO:teuthology.orchestra.run.smithi070.stdout:4826 punch from 0x96df575 to 0x96e43e0, (0x4e6b bytes) 2024-09-16T11:44:20.475 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.475 INFO:teuthology.orchestra.run.smithi070.stdout:4831 writesame 0x6666353 thru 0x667467e data_size 0x86(0xe32c bytes) 2024-09-16T11:44:20.480 INFO:teuthology.orchestra.run.smithi070.stdout:4833 write 0x8b93227 thru 0x8b98867 (0x5641 bytes) 2024-09-16T11:44:20.485 INFO:teuthology.orchestra.run.smithi070.stdout:4837 writesame 0xbf1f06a thru 0xbf25bc0 data_size 0x1(0x6b57 bytes) 2024-09-16T11:44:20.499 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.499 INFO:teuthology.orchestra.run.smithi070.stdout:4839 writesame 0xafecee6 thru 0xafee81d data_size 0x2(0x1938 bytes) 2024-09-16T11:44:20.503 INFO:teuthology.orchestra.run.smithi070.stdout:4840 writesame 0x75d7dee thru 0x75e6d85 data_size 0xa4(0xef98 bytes) 2024-09-16T11:44:20.509 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.510 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:20.508+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:20.522 INFO:teuthology.orchestra.run.smithi070.stdout:4846 trunc from 0xd007ae9 to 0x20db0f9 2024-09-16T11:44:20.524 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:20.524 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:20.524 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:20.524 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:20.533 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.535 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:20.547 INFO:teuthology.orchestra.run.smithi070.stdout:4848 writesame 0x91e3df1 thru 0x91e8c55 data_size 0x2f(0x4e65 bytes) 2024-09-16T11:44:20.547 INFO:teuthology.orchestra.run.smithi070.stdout:4849 write 0x5eae144 thru 0x5ebb566 (0xd423 bytes) 2024-09-16T11:44:20.551 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.551 INFO:teuthology.orchestra.run.smithi070.stdout:4851 read 0x4bd9316 thru 0x4be38fb (0xa5e6 bytes) 2024-09-16T11:44:20.551 INFO:teuthology.orchestra.run.smithi070.stdout:4852 read 0x2caf151 thru 0x2cb595f (0x680f bytes) 2024-09-16T11:44:20.551 INFO:teuthology.orchestra.run.smithi070.stdout:4857 punch from 0x6833327 to 0x683b55e, (0x8237 bytes) 2024-09-16T11:44:20.556 INFO:teuthology.orchestra.run.smithi070.stdout:4858 trunc from 0x91e8c56 to 0xd7573d7 2024-09-16T11:44:20.561 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.562 INFO:teuthology.orchestra.run.smithi070.stdout:4860 writesame 0xbd944a2 thru 0xbd95f52 data_size 0x1(0x1ab1 bytes) 2024-09-16T11:44:20.567 INFO:teuthology.orchestra.run.smithi070.stdout:4862 punch from 0x291c17c to 0x2929d34, (0xdbb8 bytes) 2024-09-16T11:44:20.567 INFO:teuthology.orchestra.run.smithi070.stdout:4863 write 0x362b26f thru 0x363aa23 (0xf7b5 bytes) 2024-09-16T11:44:20.571 INFO:teuthology.orchestra.run.smithi070.stdout:4864 read 0x8cc17b8 thru 0x8cc4583 (0x2dcc bytes) 2024-09-16T11:44:20.575 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:20.575 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:20.582 INFO:teuthology.orchestra.run.smithi070.stdout:4866 trunc from 0xd7573d7 to 0x3ee6045 2024-09-16T11:44:20.594 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:20.594 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:20.599 INFO:teuthology.orchestra.run.smithi070.stdout:4867 write 0x5bea785 thru 0x5bec87d (0x20f9 bytes) 2024-09-16T11:44:20.602 INFO:teuthology.orchestra.run.smithi070.stdout:4868 read 0xa09ec8 thru 0xa18c39 (0xed72 bytes) 2024-09-16T11:44:20.611 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:20.609+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:20.611 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:20.611 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:20.615 INFO:teuthology.orchestra.run.smithi070.stdout:4874 write 0xc82a0f1 thru 0xc83331d (0x922d bytes) 2024-09-16T11:44:20.619 INFO:teuthology.orchestra.run.smithi070.stdout:4875 write 0x28c2f36 thru 0x28c61c9 (0x3294 bytes) 2024-09-16T11:44:20.623 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.623 INFO:teuthology.orchestra.run.smithi070.stdout:4877 read 0xc1dd7a7 thru 0xc1e4b0d (0x7367 bytes) 2024-09-16T11:44:20.623 INFO:teuthology.orchestra.run.smithi070.stdout:4878 read 0x87a3d1 thru 0x87d137 (0x2d67 bytes) 2024-09-16T11:44:20.623 INFO:teuthology.orchestra.run.smithi070.stdout:4879 read 0x8332566 thru 0x83363c9 (0x3e64 bytes) 2024-09-16T11:44:20.623 INFO:teuthology.orchestra.run.smithi070.stdout:4882 read 0xa86271e thru 0xa8702f8 (0xdbdb bytes) 2024-09-16T11:44:20.623 INFO:teuthology.orchestra.run.smithi070.stdout:4884 write 0x77f2ec1 thru 0x77fe108 (0xb248 bytes) 2024-09-16T11:44:20.626 INFO:teuthology.orchestra.run.smithi070.stdout:4885 read 0x4cd37ef thru 0x4cdf411 (0xbc23 bytes) 2024-09-16T11:44:20.627 INFO:teuthology.orchestra.run.smithi070.stdout:4887 read 0x34f3fd9 thru 0x3503adf (0xfb07 bytes) 2024-09-16T11:44:20.627 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.630 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:20.630 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:20.636 INFO:teuthology.orchestra.run.smithi070.stdout:4890 trunc from 0xc83331e to 0x4c6b4a6 2024-09-16T11:44:20.646 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.648 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:20.648 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:20.651 INFO:teuthology.orchestra.run.smithi070.stdout:4893 write 0x558d6dd thru 0x5599953 (0xc277 bytes) 2024-09-16T11:44:20.655 INFO:teuthology.orchestra.run.smithi070.stdout:4894 read 0x15bf363 thru 0x15c355a (0x41f8 bytes) 2024-09-16T11:44:20.655 INFO:teuthology.orchestra.run.smithi070.stdout:4895 read 0x4b8150c thru 0x4b8e563 (0xd058 bytes) 2024-09-16T11:44:20.655 INFO:teuthology.orchestra.run.smithi070.stdout:4896 read 0x4a58c47 thru 0x4a5ffe9 (0x73a3 bytes) 2024-09-16T11:44:20.656 INFO:teuthology.orchestra.run.smithi070.stdout:4897 read 0x49b74c2 thru 0x49c4e5b (0xd99a bytes) 2024-09-16T11:44:20.661 INFO:teuthology.orchestra.run.smithi070.stdout:4898 trunc from 0x5599954 to 0x29df5d 2024-09-16T11:44:20.666 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:20.666 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:20.667 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:20.667 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:20.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:20.683 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:20.684 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:20.690 INFO:teuthology.orchestra.run.smithi070.stdout:4899 write 0xa50bfdd thru 0xa50e37e (0x23a2 bytes) 2024-09-16T11:44:20.693 INFO:teuthology.orchestra.run.smithi070.stdout:4902 read 0x6ff1df4 thru 0x6ff7006 (0x5213 bytes) 2024-09-16T11:44:20.693 INFO:teuthology.orchestra.run.smithi070.stdout:4903 write 0x4c46ff2 thru 0x4c4c39f (0x53ae bytes) 2024-09-16T11:44:20.698 INFO:teuthology.orchestra.run.smithi070.stdout:4904 trunc from 0xa50e37f to 0x94b8df1 2024-09-16T11:44:20.701 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:20.701 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:20.707 INFO:teuthology.orchestra.run.smithi070.stdout:4905 trunc from 0x94b8df1 to 0x9919581 2024-09-16T11:44:20.712 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:20.710+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:20.712 INFO:teuthology.orchestra.run.smithi070.stdout:4906 punch from 0x3c2ccae to 0x3c301d0, (0x3522 bytes) 2024-09-16T11:44:20.723 INFO:teuthology.orchestra.run.smithi070.stdout:4907 trunc from 0x9919581 to 0x8a56f9 2024-09-16T11:44:20.752 INFO:teuthology.orchestra.run.smithi070.stdout:4908 write 0xd4eac5b thru 0xd4f9a23 (0xedc9 bytes) 2024-09-16T11:44:20.756 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.756 INFO:teuthology.orchestra.run.smithi070.stdout:4910 punch from 0x46ddc64 to 0x46eabc9, (0xcf65 bytes) 2024-09-16T11:44:20.756 INFO:teuthology.orchestra.run.smithi070.stdout:4911 read 0x94c1baf thru 0x94c2064 (0x4b6 bytes) 2024-09-16T11:44:20.763 INFO:teuthology.orchestra.run.smithi070.stdout:4912 writesame 0xe668648 thru 0xe6752bd data_size 0x1(0xcc76 bytes) 2024-09-16T11:44:20.763 INFO:teuthology.orchestra.run.smithi070.stdout:4913 write 0x4e4ef79 thru 0x4e5970c (0xa794 bytes) 2024-09-16T11:44:20.775 INFO:teuthology.orchestra.run.smithi070.stdout:4917 trunc from 0xe6752be to 0x7120ad5 2024-09-16T11:44:20.783 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.787 INFO:teuthology.orchestra.run.smithi070.stdout:4919 trunc from 0x7120ad5 to 0x99e2431 2024-09-16T11:44:20.792 INFO:teuthology.orchestra.run.smithi070.stdout:4920 writesame 0x893d5d6 thru 0x8947ede data_size 0x1(0xa909 bytes) 2024-09-16T11:44:20.799 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:20.799 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:20.810 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:20.812 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:20.810+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:20.815 INFO:teuthology.orchestra.run.smithi070.stdout:4922 write 0x30c89ea thru 0x30cbe19 (0x3430 bytes) 2024-09-16T11:44:20.818 INFO:teuthology.orchestra.run.smithi070.stdout:4924 read 0x47487e7 thru 0x474f396 (0x6bb0 bytes) 2024-09-16T11:44:20.824 INFO:teuthology.orchestra.run.smithi070.stdout:4926 trunc from 0x99e2431 to 0xe6c1293 2024-09-16T11:44:20.829 INFO:teuthology.orchestra.run.smithi070.stdout:4927 read 0x1e6ef43 thru 0x1e6ef95 (0x53 bytes) 2024-09-16T11:44:20.829 INFO:teuthology.orchestra.run.smithi070.stdout:4928 punch from 0x80f4252 to 0x81028d2, (0xe680 bytes) 2024-09-16T11:44:20.829 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.829 INFO:teuthology.orchestra.run.smithi070.stdout:4930 read 0x3438d9b thru 0x344098e (0x7bf4 bytes) 2024-09-16T11:44:20.829 INFO:teuthology.orchestra.run.smithi070.stdout:4931 writesame 0x7ac17ec thru 0x7ac5c4c data_size 0xf(0x4461 bytes) 2024-09-16T11:44:20.839 INFO:teuthology.orchestra.run.smithi070.stdout:4933 trunc from 0xe6c1293 to 0x80354f1 2024-09-16T11:44:20.849 INFO:teuthology.orchestra.run.smithi070.stdout:4934 read 0x1610e24 thru 0x161c9ca (0xbba7 bytes) 2024-09-16T11:44:20.849 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.849 INFO:teuthology.orchestra.run.smithi070.stdout:4936 writesame 0x5be6d4a thru 0x5bf162b data_size 0x1(0xa8e2 bytes) 2024-09-16T11:44:20.870 INFO:teuthology.orchestra.run.smithi070.stdout:4938 read 0x6a8c87a thru 0x6a9590e (0x9095 bytes) 2024-09-16T11:44:20.874 INFO:teuthology.orchestra.run.smithi070.stdout:4939 trunc from 0x80354f1 to 0xb62b43c 2024-09-16T11:44:20.880 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.880 INFO:teuthology.orchestra.run.smithi070.stdout:4941 read 0xb0d7189 thru 0xb0e19d1 (0xa849 bytes) 2024-09-16T11:44:20.880 INFO:teuthology.orchestra.run.smithi070.stdout:4942 write 0x33616a5 thru 0x336c516 (0xae72 bytes) 2024-09-16T11:44:20.883 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.883 INFO:teuthology.orchestra.run.smithi070.stdout:4944 write 0x492d5b7 thru 0x4939b2e (0xc578 bytes) 2024-09-16T11:44:20.887 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.888 INFO:teuthology.orchestra.run.smithi070.stdout:4949 write 0xb3201de thru 0xb32a75a (0xa57d bytes) 2024-09-16T11:44:20.891 INFO:teuthology.orchestra.run.smithi070.stdout:4950 punch from 0x4c02c6 to 0x4c46a8, (0x43e2 bytes) 2024-09-16T11:44:20.891 INFO:teuthology.orchestra.run.smithi070.stdout:4951 punch from 0x3afda29 to 0x3b0637b, (0x8952 bytes) 2024-09-16T11:44:20.892 INFO:teuthology.orchestra.run.smithi070.stdout:4952 trunc from 0xb62b43c to 0xbe2287f 2024-09-16T11:44:20.903 INFO:teuthology.orchestra.run.smithi070.stdout:4953 write 0xd0cf02b thru 0xd0d69ff (0x79d5 bytes) 2024-09-16T11:44:20.906 INFO:teuthology.orchestra.run.smithi070.stdout:4954 writesame 0x431faa9 thru 0x43281db data_size 0x3(0x8733 bytes) 2024-09-16T11:44:20.913 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:20.910+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:20.914 INFO:teuthology.orchestra.run.smithi070.stdout:4955 read 0xa027778 thru 0xa027fa3 (0x82c bytes) 2024-09-16T11:44:20.920 INFO:teuthology.orchestra.run.smithi070.stdout:4958 write 0xdf06a0a thru 0xdf169de (0xffd5 bytes) 2024-09-16T11:44:20.924 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:20.924 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:20.926 INFO:teuthology.orchestra.run.smithi070.stdout:4959 read 0x7fb39b6 thru 0x7fc01a9 (0xc7f4 bytes) 2024-09-16T11:44:20.926 INFO:teuthology.orchestra.run.smithi070.stdout:4962 read 0x90bda82 thru 0x90c224a (0x47c9 bytes) 2024-09-16T11:44:20.931 INFO:teuthology.orchestra.run.smithi070.stdout:4963 writesame 0xe1754f4 thru 0xe17be80 data_size 0x1(0x698d bytes) 2024-09-16T11:44:20.933 INFO:teuthology.orchestra.run.smithi070.stdout:4964 trunc from 0xe17be81 to 0xc44063a 2024-09-16T11:44:20.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:20.942 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.943 INFO:teuthology.orchestra.run.smithi070.stdout:4967 writesame 0xb002161 thru 0xb00a527 data_size 0x41(0x83c7 bytes) 2024-09-16T11:44:20.948 INFO:teuthology.orchestra.run.smithi070.stdout:4970 read 0x4feca14 thru 0x4ff3fc1 (0x75ae bytes) 2024-09-16T11:44:20.948 INFO:teuthology.orchestra.run.smithi070.stdout:4971 read 0x9eeb8f7 thru 0x9eee36e (0x2a78 bytes) 2024-09-16T11:44:20.948 INFO:teuthology.orchestra.run.smithi070.stdout:4972 read 0x4c24081 thru 0x4c24e90 (0xe10 bytes) 2024-09-16T11:44:20.948 INFO:teuthology.orchestra.run.smithi070.stdout:4973 punch from 0x2342b65 to 0x234866f, (0x5b0a bytes) 2024-09-16T11:44:20.949 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.959 INFO:teuthology.orchestra.run.smithi070.stdout:4977 trunc from 0xc44063a to 0x2f1ec90 2024-09-16T11:44:20.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:44:20.971 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.971 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:20.972 INFO:teuthology.orchestra.run.smithi070.stdout:4981 punch from 0x16d74d1 to 0x16e3171, (0xbca0 bytes) 2024-09-16T11:44:20.977 INFO:teuthology.orchestra.run.smithi070.stdout:4983 write 0x3432702 thru 0x3440e2a (0xe729 bytes) 2024-09-16T11:44:20.984 INFO:teuthology.orchestra.run.smithi070.stdout:4985 write 0x131ee37 thru 0x132a3aa (0xb574 bytes) 2024-09-16T11:44:20.990 INFO:teuthology.orchestra.run.smithi070.stdout:4986 punch from 0x126be71 to 0x127af3d, (0xf0cc bytes) 2024-09-16T11:44:20.991 INFO:teuthology.orchestra.run.smithi070.stdout:4987 read 0xb2d2f0 thru 0xb3cd9e (0xfaaf bytes) 2024-09-16T11:44:20.991 INFO:teuthology.orchestra.run.smithi070.stdout:4988 write 0x20997c1 thru 0x209be9c (0x26dc bytes) 2024-09-16T11:44:20.993 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.012 INFO:teuthology.orchestra.run.smithi070.stdout:4990 write 0xec372b1 thru 0xec43981 (0xc6d1 bytes) 2024-09-16T11:44:21.013 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:21.011+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:21.017 INFO:teuthology.orchestra.run.smithi070.stdout:4991 punch from 0x3e57b1e to 0x3e6138a, (0x986c bytes) 2024-09-16T11:44:21.017 INFO:teuthology.orchestra.run.smithi070.stdout:4992 read 0xb3062e8 thru 0xb30c434 (0x614d bytes) 2024-09-16T11:44:21.017 INFO:teuthology.orchestra.run.smithi070.stdout:4997 punch from 0xffdd38 to 0x100bc4f, (0xdf17 bytes) 2024-09-16T11:44:21.017 INFO:teuthology.orchestra.run.smithi070.stdout:4998 punch from 0x8894603 to 0x88997ac, (0x51a9 bytes) 2024-09-16T11:44:21.017 INFO:teuthology.orchestra.run.smithi070.stdout:5000 punch from 0x3a6b53d to 0x3a72af5, (0x75b8 bytes) 2024-09-16T11:44:21.017 INFO:teuthology.orchestra.run.smithi070.stdout:5002 read 0x406b9f3 thru 0x407b3bc (0xf9ca bytes) 2024-09-16T11:44:21.018 INFO:teuthology.orchestra.run.smithi070.stdout:5004 punch from 0x69a56c3 to 0x69b075f, (0xb09c bytes) 2024-09-16T11:44:21.018 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.018 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.018 INFO:teuthology.orchestra.run.smithi070.stdout:5007 writesame 0x777dae7 thru 0x778cc06 data_size 0x30(0xf120 bytes) 2024-09-16T11:44:21.021 INFO:teuthology.orchestra.run.smithi070.stdout:5008 write 0x418d90 thru 0x41f96c (0x6bdd bytes) 2024-09-16T11:44:21.024 INFO:teuthology.orchestra.run.smithi070.stdout:5009 read 0x689146a thru 0x689e29e (0xce35 bytes) 2024-09-16T11:44:21.033 INFO:teuthology.orchestra.run.smithi070.stdout:5010 trunc from 0xec43982 to 0x7665495 2024-09-16T11:44:21.051 INFO:teuthology.orchestra.run.smithi070.stdout:5012 trunc from 0x7665495 to 0x1009f3c 2024-09-16T11:44:21.060 INFO:teuthology.orchestra.run.smithi070.stdout:5017 read 0x5854ee thru 0x58ef68 (0x9a7b bytes) 2024-09-16T11:44:21.060 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:21.060 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:21.060 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.061 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.061 INFO:teuthology.orchestra.run.smithi070.stdout:5022 punch from 0xd989b1 to 0xd9aa6a, (0x20b9 bytes) 2024-09-16T11:44:21.061 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.068 INFO:teuthology.orchestra.run.smithi070.stdout:5024 write 0x37ab32c thru 0x37b56d2 (0xa3a7 bytes) 2024-09-16T11:44:21.070 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:21.072 INFO:teuthology.orchestra.run.smithi070.stdout:5026 read 0x3324f27 thru 0x33311d5 (0xc2af bytes) 2024-09-16T11:44:21.072 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.081 INFO:teuthology.orchestra.run.smithi070.stdout:5028 write 0x708cd2b thru 0x7092632 (0x5908 bytes) 2024-09-16T11:44:21.093 INFO:teuthology.orchestra.run.smithi070.stdout:5029 writesame 0xa0c8af3 thru 0xa0cd318 data_size 0x5(0x4826 bytes) 2024-09-16T11:44:21.104 INFO:teuthology.orchestra.run.smithi070.stdout:5030 trunc from 0xa0cd319 to 0xd05abd 2024-09-16T11:44:21.113 INFO:teuthology.orchestra.run.smithi070.stdout:5031 read 0x2195c thru 0x25814 (0x3eb9 bytes) 2024-09-16T11:44:21.114 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:21.111+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:21.123 INFO:teuthology.orchestra.run.smithi070.stdout:5032 trunc from 0xd05abd to 0x8c689c4 2024-09-16T11:44:21.140 INFO:teuthology.orchestra.run.smithi070.stdout:5033 writesame 0xdd86050 thru 0xdd90977 data_size 0x4(0xa928 bytes) 2024-09-16T11:44:21.140 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.140 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.150 INFO:teuthology.orchestra.run.smithi070.stdout:5038 trunc from 0xdd90978 to 0x5793992 2024-09-16T11:44:21.155 INFO:teuthology.orchestra.run.smithi070.stdout:5039 punch from 0x5301cc to 0x530d75, (0xba9 bytes) 2024-09-16T11:44:21.155 INFO:teuthology.orchestra.run.smithi070.stdout:5041 read 0x464ea78 thru 0x4654684 (0x5c0d bytes) 2024-09-16T11:44:21.156 INFO:teuthology.orchestra.run.smithi070.stdout:5042 read 0x5656c69 thru 0x566076a (0x9b02 bytes) 2024-09-16T11:44:21.158 INFO:teuthology.orchestra.run.smithi070.stdout:5043 trunc from 0x5793992 to 0x83f11f1 2024-09-16T11:44:21.163 INFO:teuthology.orchestra.run.smithi070.stdout:5045 read 0x6ae0642 thru 0x6ae3b93 (0x3552 bytes) 2024-09-16T11:44:21.164 INFO:teuthology.orchestra.run.smithi070.stdout:5046 punch from 0x33d69b2 to 0x33e38db, (0xcf29 bytes) 2024-09-16T11:44:21.169 INFO:teuthology.orchestra.run.smithi070.stdout:5047 write 0x8e7ee95 thru 0x8e81819 (0x2985 bytes) 2024-09-16T11:44:21.173 INFO:teuthology.orchestra.run.smithi070.stdout:5048 read 0x4ecc71a thru 0x4ed2982 (0x6269 bytes) 2024-09-16T11:44:21.174 INFO:teuthology.orchestra.run.smithi070.stdout:5049 read 0x68e7d28 thru 0x68ebe42 (0x411b bytes) 2024-09-16T11:44:21.174 INFO:teuthology.orchestra.run.smithi070.stdout:5050 punch from 0x1aad869 to 0x1abb77f, (0xdf16 bytes) 2024-09-16T11:44:21.174 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.174 INFO:teuthology.orchestra.run.smithi070.stdout:5052 write 0x2160623 thru 0x216e06e (0xda4c bytes) 2024-09-16T11:44:21.177 INFO:teuthology.orchestra.run.smithi070.stdout:5053 punch from 0x878aebe to 0x878b7e8, (0x92a bytes) 2024-09-16T11:44:21.177 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.177 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.177 INFO:teuthology.orchestra.run.smithi070.stdout:5056 write 0x53494d2 thru 0x5357677 (0xe1a6 bytes) 2024-09-16T11:44:21.190 INFO:teuthology.orchestra.run.smithi070.stdout:5057 write 0xc4a9aac thru 0xc4acd55 (0x32aa bytes) 2024-09-16T11:44:21.191 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:21.191 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:21.192 INFO:teuthology.orchestra.run.smithi070.stdout:5058 writesame 0x502a31c thru 0x50389db data_size 0x8e(0xe6c0 bytes) 2024-09-16T11:44:21.201 INFO:teuthology.orchestra.run.smithi070.stdout:5059 trunc from 0xc4acd56 to 0x8324105 2024-09-16T11:44:21.202 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:21.210 INFO:teuthology.orchestra.run.smithi070.stdout:5062 read 0x6c34618 thru 0x6c42318 (0xdd01 bytes) 2024-09-16T11:44:21.210 INFO:teuthology.orchestra.run.smithi070.stdout:5064 punch from 0x550723f to 0x55130f6, (0xbeb7 bytes) 2024-09-16T11:44:21.210 INFO:teuthology.orchestra.run.smithi070.stdout:5065 punch from 0x2235d20 to 0x2245bbf, (0xfe9f bytes) 2024-09-16T11:44:21.214 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:21.212+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:21.219 INFO:teuthology.orchestra.run.smithi070.stdout:5066 write 0xb958b26 thru 0xb968a99 (0xff74 bytes) 2024-09-16T11:44:21.225 INFO:teuthology.orchestra.run.smithi070.stdout:5067 write 0x18477a0 thru 0x185691c (0xf17d bytes) 2024-09-16T11:44:21.235 INFO:teuthology.orchestra.run.smithi070.stdout:5068 writesame 0xc121524 thru 0xc1232d5 data_size 0x15(0x1db2 bytes) 2024-09-16T11:44:21.235 INFO:teuthology.orchestra.run.smithi070.stdout:5069 read 0x81b976f thru 0x81bb848 (0x20da bytes) 2024-09-16T11:44:21.235 INFO:teuthology.orchestra.run.smithi070.stdout:5070 punch from 0x52f57b6 to 0x52fa572, (0x4dbc bytes) 2024-09-16T11:44:21.235 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.240 INFO:teuthology.orchestra.run.smithi070.stdout:5073 write 0xc3f2afc thru 0xc3f5bf7 (0x30fc bytes) 2024-09-16T11:44:21.243 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.243 INFO:teuthology.orchestra.run.smithi070.stdout:5076 write 0x80b029f thru 0x80b751e (0x7280 bytes) 2024-09-16T11:44:21.251 INFO:teuthology.orchestra.run.smithi070.stdout:5077 write 0xcbcf0bb thru 0xcbd4b1e (0x5a64 bytes) 2024-09-16T11:44:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:5079 punch from 0x66d5d1f to 0x66e294f, (0xcc30 bytes) 2024-09-16T11:44:21.257 INFO:teuthology.orchestra.run.smithi070.stdout:5081 write 0x9b834e7 thru 0x9b8e83d (0xb357 bytes) 2024-09-16T11:44:21.261 INFO:teuthology.orchestra.run.smithi070.stdout:5082 punch from 0x8d3bbf3 to 0x8d43fe6, (0x83f3 bytes) 2024-09-16T11:44:21.261 INFO:teuthology.orchestra.run.smithi070.stdout:5083 read 0x67f64ab thru 0x67feb7f (0x86d5 bytes) 2024-09-16T11:44:21.261 INFO:teuthology.orchestra.run.smithi070.stdout:5084 read 0x2bf0d50 thru 0x2bff068 (0xe319 bytes) 2024-09-16T11:44:21.261 INFO:teuthology.orchestra.run.smithi070.stdout:5085 read 0xbb0bddb thru 0xbb0d459 (0x167f bytes) 2024-09-16T11:44:21.262 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.262 INFO:teuthology.orchestra.run.smithi070.stdout:5087 write 0x3874255 thru 0x387ff5a (0xbd06 bytes) 2024-09-16T11:44:21.282 INFO:teuthology.orchestra.run.smithi070.stdout:5089 trunc from 0xcbd4b1f to 0xed936a 2024-09-16T11:44:21.294 INFO:teuthology.orchestra.run.smithi070.stdout:5090 read 0x5bb8b7 thru 0x5c0e4a (0x5594 bytes) 2024-09-16T11:44:21.294 INFO:teuthology.orchestra.run.smithi070.stdout:5091 read 0x503cee thru 0x5081fa (0x450d bytes) 2024-09-16T11:44:21.294 INFO:teuthology.orchestra.run.smithi070.stdout:5092 read 0x14e26b thru 0x157662 (0x93f8 bytes) 2024-09-16T11:44:21.295 INFO:teuthology.orchestra.run.smithi070.stdout:5093 punch from 0x2aa148 to 0x2b897e, (0xe836 bytes) 2024-09-16T11:44:21.299 INFO:teuthology.orchestra.run.smithi070.stdout:5094 trunc from 0xed936a to 0x476a72b 2024-09-16T11:44:21.304 INFO:teuthology.orchestra.run.smithi070.stdout:5097 read 0x177e2c2 thru 0x17858fb (0x763a bytes) 2024-09-16T11:44:21.304 INFO:teuthology.orchestra.run.smithi070.stdout:5098 punch from 0x3492a98 to 0x349c93b, (0x9ea3 bytes) 2024-09-16T11:44:21.314 INFO:teuthology.orchestra.run.smithi070.stdout:5099 writesame 0x82b8176 thru 0x82beeac data_size 0x1(0x6d37 bytes) 2024-09-16T11:44:21.314 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.314 INFO:teuthology.orchestra.run.smithi070.stdout:5101 write 0x4df5af9 thru 0x4e01511 (0xba19 bytes) 2024-09-16T11:44:21.315 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:21.312+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:21.323 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:21.323 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:21.330 INFO:teuthology.orchestra.run.smithi070.stdout:5102 trunc from 0x82beead to 0x9f9eef 2024-09-16T11:44:21.336 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:21.339 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.339 INFO:teuthology.orchestra.run.smithi070.stdout:5104 writesame 0x9e2e7c thru 0x9e57a1 data_size 0x17(0x2926 bytes) 2024-09-16T11:44:21.342 INFO:teuthology.orchestra.run.smithi070.stdout:5105 read 0x6677a8 thru 0x673375 (0xbbce bytes) 2024-09-16T11:44:21.343 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:21.343 INFO:teuthology.orchestra.run.smithi070.stdout:5107 punch from 0x97426a to 0x97f156, (0xaeec bytes) 2024-09-16T11:44:21.343 INFO:teuthology.orchestra.run.smithi070.stdout:5108 read 0x93b5a9 thru 0x93ebcf (0x3627 bytes) 2024-09-16T11:44:21.343 INFO:teuthology.orchestra.run.smithi070.stdout:5109 punch from 0x5ab95d to 0x5b95f4, (0xdc97 bytes) 2024-09-16T11:44:21.360 INFO:teuthology.orchestra.run.smithi070.stdout:5111 writesame 0xaefd78f thru 0xaf07a33 data_size 0x1(0xa2a5 bytes) 2024-09-16T11:44:21.371 INFO:teuthology.orchestra.run.smithi070.stdout:5112 trunc from 0xaf07a34 to 0x10b6bcc 2024-09-16T11:44:21.387 INFO:teuthology.orchestra.run.smithi070.stdout:5113 writesame 0x486a258 thru 0x487162a data_size 0x1(0x73d3 bytes) 2024-09-16T11:44:21.399 INFO:teuthology.orchestra.run.smithi070.stdout:5114 write 0xa1c430d thru 0xa1ce154 (0x9e48 bytes) 2024-09-16T11:44:21.404 INFO:teuthology.orchestra.run.smithi070.stdout:5115 clone 32 order 20 su 262144 sc 13 2024-09-16T11:44:21.416 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:21.414+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:21.446 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:21.446 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:21.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:21.516 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:21.514+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:21.593 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:21.593 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:21.601 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:21.645 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:21.643+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:21.700 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:21.700 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:21.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:21.718 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:21.716+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:21.818 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:21.816+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:21.846 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:21.846 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:21.854 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:21.938 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:21.936+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:21.962 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T11:44:22.000 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:22.001 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:22.011 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:22.052 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:22.050+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:22.111 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:22.111 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:22.119 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:22.121 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:22.118+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:22.127 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-16T11:44:22.132 INFO:teuthology.orchestra.run.smithi070.stdout:5117 trunc from 0xa1ce155 to 0xb5b55ed 2024-09-16T11:44:22.139 INFO:teuthology.orchestra.run.smithi070.stdout:5118 read 0x6f806e5 thru 0x6f88b1f (0x843b bytes) 2024-09-16T11:44:22.139 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:22.139 INFO:teuthology.orchestra.run.smithi070.stdout:5122 read 0x5c38ce8 thru 0x5c45fc4 (0xd2dd bytes) 2024-09-16T11:44:22.139 INFO:teuthology.orchestra.run.smithi070.stdout:5124 read 0xab49551 thru 0xab4e468 (0x4f18 bytes) 2024-09-16T11:44:22.148 INFO:teuthology.orchestra.run.smithi070.stdout:5125 trunc from 0xb5b55ed to 0x3ab897b 2024-09-16T11:44:22.168 INFO:teuthology.orchestra.run.smithi070.stdout:truncating to largest ever: 0xed8c5ad 2024-09-16T11:44:22.181 INFO:teuthology.orchestra.run.smithi070.stdout:5126 trunc from 0x3ab897b to 0xed8c5ad 2024-09-16T11:44:22.187 INFO:teuthology.orchestra.run.smithi070.stdout:5127 read 0x9f5129a thru 0x9f5f693 (0xe3fa bytes) 2024-09-16T11:44:22.187 INFO:teuthology.orchestra.run.smithi070.stdout:5128 read 0xe4d77ba thru 0xe4da312 (0x2b59 bytes) 2024-09-16T11:44:22.187 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:22.187 INFO:teuthology.orchestra.run.smithi070.stdout:5130 write 0x85c61ed thru 0x85cd08b (0x6e9f bytes) 2024-09-16T11:44:22.190 INFO:teuthology.orchestra.run.smithi070.stdout:5131 writesame 0x20f6bb6 thru 0x20f8253 data_size 0x5(0x169e bytes) 2024-09-16T11:44:22.193 INFO:teuthology.orchestra.run.smithi070.stdout:5133 writesame 0x920d7b9 thru 0x921a0a6 data_size 0x3(0xc8ee bytes) 2024-09-16T11:44:22.203 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:22.203 INFO:teuthology.orchestra.run.smithi070.stdout:5135 read 0xec74a66 thru 0xec848d2 (0xfe6d bytes) 2024-09-16T11:44:22.203 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:22.203 INFO:teuthology.orchestra.run.smithi070.stdout:5137 read 0x8a3cbbc thru 0x8a47181 (0xa5c6 bytes) 2024-09-16T11:44:22.203 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:22.203 INFO:teuthology.orchestra.run.smithi070.stdout:5140 write 0x6c093c4 thru 0x6c0cde7 (0x3a24 bytes) 2024-09-16T11:44:22.221 INFO:teuthology.orchestra.run.smithi070.stdout:5141 trunc from 0xed8c5ad to 0x194dfb6 2024-09-16T11:44:22.222 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:22.219+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:22.225 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:22.225 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:22.235 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:22.236 INFO:teuthology.orchestra.run.smithi070.stdout:5142 read 0xd1612b thru 0xd1f850 (0x9726 bytes) 2024-09-16T11:44:22.236 INFO:teuthology.orchestra.run.smithi070.stdout:5143 read 0x156b9f thru 0x15f301 (0x8763 bytes) 2024-09-16T11:44:22.237 INFO:teuthology.orchestra.run.smithi070.stdout:5144 punch from 0x114522 to 0x11e48a, (0x9f68 bytes) 2024-09-16T11:44:22.241 INFO:teuthology.orchestra.run.smithi070.stdout:5146 trunc from 0x194dfb6 to 0x57cd13f 2024-09-16T11:44:22.246 INFO:teuthology.orchestra.run.smithi070.stdout:5148 write 0x1ddaea0 thru 0x1de0653 (0x57b4 bytes) 2024-09-16T11:44:22.259 INFO:teuthology.orchestra.run.smithi070.stdout:5149 writesame 0x88aeb53 thru 0x88b0465 data_size 0x1(0x1913 bytes) 2024-09-16T11:44:22.260 INFO:teuthology.orchestra.run.smithi070.stdout:5150 punch from 0x1a40059 to 0x1a49682, (0x9629 bytes) 2024-09-16T11:44:22.260 INFO:teuthology.orchestra.run.smithi070.stdout:5151 read 0x6a19a7f thru 0x6a25a83 (0xc005 bytes) 2024-09-16T11:44:22.260 INFO:teuthology.orchestra.run.smithi070.stdout:5152 punch from 0x7f91020 to 0x7f9e536, (0xd516 bytes) 2024-09-16T11:44:22.269 INFO:teuthology.orchestra.run.smithi070.stdout:5153 writesame 0xcb3276e thru 0xcb3b7f5 data_size 0x32(0x9088 bytes) 2024-09-16T11:44:22.276 INFO:teuthology.orchestra.run.smithi070.stdout:5154 write 0xe2df033 thru 0xe2e6aa7 (0x7a75 bytes) 2024-09-16T11:44:22.278 INFO:teuthology.orchestra.run.smithi070.stdout:5155 read 0x2ec805c thru 0x2ecb1af (0x3154 bytes) 2024-09-16T11:44:22.278 INFO:teuthology.orchestra.run.smithi070.stdout:5156 write 0x5c8f8cd thru 0x5c918e0 (0x2014 bytes) 2024-09-16T11:44:22.281 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:22.287 INFO:teuthology.orchestra.run.smithi070.stdout:5161 writesame 0xed4ca3a thru 0xed4d99a data_size 0x1(0xf61 bytes) 2024-09-16T11:44:22.287 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:22.287 INFO:teuthology.orchestra.run.smithi070.stdout:5164 writesame 0xacc367 thru 0xacee3b data_size 0x33(0x2ad5 bytes) 2024-09-16T11:44:22.291 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:22.296 INFO:teuthology.orchestra.run.smithi070.stdout:5166 trunc from 0xed4d99b to 0xac28377 2024-09-16T11:44:22.304 INFO:teuthology.orchestra.run.smithi070.stdout:5169 write 0xa010ab5 thru 0xa01a03e (0x958a bytes) 2024-09-16T11:44:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:5170 read 0x2d9aa30 thru 0x2da1353 (0x6924 bytes) 2024-09-16T11:44:22.309 INFO:teuthology.orchestra.run.smithi070.stdout:5171 trunc from 0xac28377 to 0xbbcc1fe 2024-09-16T11:44:22.314 INFO:teuthology.orchestra.run.smithi070.stdout:5172 read 0x78eeb6a thru 0x78f54fd (0x6994 bytes) 2024-09-16T11:44:22.314 INFO:teuthology.orchestra.run.smithi070.stdout:5173 read 0x2ee646e thru 0x2ef4899 (0xe42c bytes) 2024-09-16T11:44:22.314 INFO:teuthology.orchestra.run.smithi070.stdout:5175 write 0xc952b9 thru 0xc9a43c (0x5184 bytes) 2024-09-16T11:44:22.317 INFO:teuthology.orchestra.run.smithi070.stdout:5176 read 0x3389e8c thru 0x339501e (0xb193 bytes) 2024-09-16T11:44:22.317 INFO:teuthology.orchestra.run.smithi070.stdout:5177 write 0x536278 thru 0x53e557 (0x82e0 bytes) 2024-09-16T11:44:22.322 INFO:teuthology.orchestra.run.smithi070.stdout:5178 write 0x13faa43 thru 0x140892c (0xdeea bytes) 2024-09-16T11:44:22.323 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:22.320+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:22.326 INFO:teuthology.orchestra.run.smithi070.stdout:5179 write 0x49c230e thru 0x49ce502 (0xc1f5 bytes) 2024-09-16T11:44:22.329 INFO:teuthology.orchestra.run.smithi070.stdout:5180 read 0x6e1c82b thru 0x6e2baed (0xf2c3 bytes) 2024-09-16T11:44:22.329 INFO:teuthology.orchestra.run.smithi070.stdout:5181 write 0x4b9c4f6 thru 0x4baa60c (0xe117 bytes) 2024-09-16T11:44:22.332 INFO:teuthology.orchestra.run.smithi070.stdout:5182 read 0x9ecf2f8 thru 0x9ed2bfe (0x3907 bytes) 2024-09-16T11:44:22.332 INFO:teuthology.orchestra.run.smithi070.stdout:5184 punch from 0x3125761 to 0x3132386, (0xcc25 bytes) 2024-09-16T11:44:22.339 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:22.339 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:22.339 INFO:teuthology.orchestra.run.smithi070.stdout:5185 trunc from 0xbbcc1fe to 0x640637f 2024-09-16T11:44:22.348 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:22.349 INFO:teuthology.orchestra.run.smithi070.stdout:5186 write 0x5e0c7b1 thru 0x5e17548 (0xad98 bytes) 2024-09-16T11:44:22.352 INFO:teuthology.orchestra.run.smithi070.stdout:5187 read 0x1e1420e thru 0x1e17058 (0x2e4b bytes) 2024-09-16T11:44:22.352 INFO:teuthology.orchestra.run.smithi070.stdout:5188 write 0x55b306 thru 0x560348 (0x5043 bytes) 2024-09-16T11:44:22.356 INFO:teuthology.orchestra.run.smithi070.stdout:5190 read 0x8d0cb5 thru 0x8e0a72 (0xfdbe bytes) 2024-09-16T11:44:22.356 INFO:teuthology.orchestra.run.smithi070.stdout:5191 punch from 0x42cedbd to 0x42d06ec, (0x192f bytes) 2024-09-16T11:44:22.356 INFO:teuthology.orchestra.run.smithi070.stdout:5193 punch from 0x177d958 to 0x177f55c, (0x1c04 bytes) 2024-09-16T11:44:22.356 INFO:teuthology.orchestra.run.smithi070.stdout:5195 read 0x416ebe3 thru 0x416ed88 (0x1a6 bytes) 2024-09-16T11:44:22.364 INFO:teuthology.orchestra.run.smithi070.stdout:5196 write 0x8902d15 thru 0x89124be (0xf7aa bytes) 2024-09-16T11:44:22.367 INFO:teuthology.orchestra.run.smithi070.stdout:5197 writesame 0x88ea0e2 thru 0x88f31e2 data_size 0x1(0x9101 bytes) 2024-09-16T11:44:22.383 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:22.383 INFO:teuthology.orchestra.run.smithi070.stdout:5199 read 0x29a7c91 thru 0x29ab811 (0x3b81 bytes) 2024-09-16T11:44:22.383 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:22.383 INFO:teuthology.orchestra.run.smithi070.stdout:5202 punch from 0x4b327a7 to 0x4b3e2fd, (0xbb56 bytes) 2024-09-16T11:44:22.383 INFO:teuthology.orchestra.run.smithi070.stdout:5203 trunc from 0x89124bf to 0x835a601 2024-09-16T11:44:22.389 INFO:teuthology.orchestra.run.smithi070.stdout:5204 read 0xf9a45f thru 0xfa5fac (0xbb4e bytes) 2024-09-16T11:44:22.391 INFO:teuthology.orchestra.run.smithi070.stdout:5205 trunc from 0x835a601 to 0x97bf61d 2024-09-16T11:44:22.395 INFO:teuthology.orchestra.run.smithi070.stdout:5206 write 0x8c6c6f9 thru 0x8c74a90 (0x8398 bytes) 2024-09-16T11:44:22.397 INFO:teuthology.orchestra.run.smithi070.stdout:5207 read 0x2d5b1ba thru 0x2d60269 (0x50b0 bytes) 2024-09-16T11:44:22.397 INFO:teuthology.orchestra.run.smithi070.stdout:5208 read 0x8c57803 thru 0x8c5a752 (0x2f50 bytes) 2024-09-16T11:44:22.397 INFO:teuthology.orchestra.run.smithi070.stdout:5209 read 0x6b063da thru 0x6b0c6e1 (0x6308 bytes) 2024-09-16T11:44:22.397 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:22.397 INFO:teuthology.orchestra.run.smithi070.stdout:5211 write 0x60f332 thru 0x612a48 (0x3717 bytes) 2024-09-16T11:44:22.407 INFO:teuthology.orchestra.run.smithi070.stdout:5214 write 0xb94b876 thru 0xb95974d (0xded8 bytes) 2024-09-16T11:44:22.410 INFO:teuthology.orchestra.run.smithi070.stdout:5215 write 0x6393b1e thru 0x639ab2f (0x7012 bytes) 2024-09-16T11:44:22.413 INFO:teuthology.orchestra.run.smithi070.stdout:5216 read 0x872bbdc thru 0x872dfc9 (0x23ee bytes) 2024-09-16T11:44:22.413 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:22.413 INFO:teuthology.orchestra.run.smithi070.stdout:5218 read 0x461ac5c thru 0x461fcdd (0x5082 bytes) 2024-09-16T11:44:22.418 INFO:teuthology.orchestra.run.smithi070.stdout:5219 trunc from 0xb95974e to 0x76dcfb5 2024-09-16T11:44:22.424 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:22.422+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:22.427 INFO:teuthology.orchestra.run.smithi070.stdout:5220 write 0x461fff3 thru 0x462ae75 (0xae83 bytes) 2024-09-16T11:44:22.431 INFO:teuthology.orchestra.run.smithi070.stdout:5221 write 0x48fce55 thru 0x4903ad7 (0x6c83 bytes) 2024-09-16T11:44:22.436 INFO:teuthology.orchestra.run.smithi070.stdout:5223 clone 33 order 25 su 524288 sc 8 2024-09-16T11:44:22.488 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:22.488 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:22.496 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:22.525 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:22.523+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:22.593 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:22.593 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:22.602 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:22.645 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:22.643+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:22.700 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:22.700 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:22.709 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:22.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:44:22.758 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:22.756+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:22.828 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:22.826+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:22.830 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:22.830 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:22.846 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:22.846 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:22.847 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:22.847 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:22.855 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:22.860 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:22.861 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:22.875 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:22.875 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:22.890 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:22.891 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:22.906 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:22.906 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:22.922 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:22.922 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:22.929 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:22.927+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:22.937 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:22.937 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:22.954 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:22.954 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:22.963 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:23.016 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone32 from 0x76dcfb5 (overlap 0x194dfb6) to 0x50f98a 2024-09-16T11:44:23.030 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:23.028+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:23.073 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19282 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:23.073 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.597627558999999, 2024-09-16T11:44:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521151000000002, 2024-09-16T11:44:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19282 2024-09-16T11:44:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:23.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472838+0000", 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472848+0000", 2024-09-16T11:44:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9799999999999993e-06 2024-09-16T11:44:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472858+0000", 2024-09-16T11:44:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9429999999999995e-06 2024-09-16T11:44:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472862+0000", 2024-09-16T11:44:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4460000000000003e-06 2024-09-16T11:44:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501277+0000", 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028414900999999999 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501322+0000", 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4880999999999999e-05 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501358+0000", 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6245000000000003e-05 2024-09-16T11:44:23.077 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19286 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7979008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.597254627000002, 2024-09-16T11:44:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521860999999999, 2024-09-16T11:44:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19286 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473211+0000", 2024-09-16T11:44:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:44:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473214+0000", 2024-09-16T11:44:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1080000000000001e-06 2024-09-16T11:44:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473226+0000", 2024-09-16T11:44:23.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2441e-05 2024-09-16T11:44:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473234+0000", 2024-09-16T11:44:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729999999999998e-06 2024-09-16T11:44:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501592+0000", 2024-09-16T11:44:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028358491999999999 2024-09-16T11:44:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501671+0000", 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8846999999999999e-05 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501732+0000", 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1118000000000007e-05 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20619 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7610368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.509905458999999, 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028500152000000001, 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20619 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:23.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560560+0000", 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9499999999999998e-07 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560561+0000", 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.071e-06 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560565+0000", 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560567+0000", 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588949+0000", 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381878999999999 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588989+0000", 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0649000000000003e-05 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589059+0000", 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0239000000000003e-05 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.086 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20620 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7716864~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.509896058999999, 2024-09-16T11:44:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028627621999999998, 2024-09-16T11:44:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.087 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20620 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562377+0000", 2024-09-16T11:44:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018081729999999999 2024-09-16T11:44:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562380+0000", 2024-09-16T11:44:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0469999999999998e-06 2024-09-16T11:44:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562386+0000", 2024-09-16T11:44:23.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2040000000000002e-06 2024-09-16T11:44:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562390+0000", 2024-09-16T11:44:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:44:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589092+0000", 2024-09-16T11:44:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026701798999999998 2024-09-16T11:44:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589139+0000", 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7320000000000001e-05 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589196+0000", 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7201999999999999e-05 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41954 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.091 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.656190269, 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028594441000000002, 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41954 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:23.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414275+0000", 2024-09-16T11:44:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:44:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414277+0000", 2024-09-16T11:44:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414282+0000", 2024-09-16T11:44:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7559999999999996e-06 2024-09-16T11:44:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414286+0000", 2024-09-16T11:44:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:44:23.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442778+0000", 2024-09-16T11:44:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028492237 2024-09-16T11:44:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442816+0000", 2024-09-16T11:44:23.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8198000000000003e-05 2024-09-16T11:44:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442869+0000", 2024-09-16T11:44:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2589000000000001e-05 2024-09-16T11:44:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41955 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.656175211000001, 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028687865, 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41955 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:44:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414292+0000", 2024-09-16T11:44:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-09-16T11:44:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414299+0000", 2024-09-16T11:44:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4279999999999999e-06 2024-09-16T11:44:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414303+0000", 2024-09-16T11:44:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:44:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442892+0000", 2024-09-16T11:44:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028589564000000001 2024-09-16T11:44:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442930+0000", 2024-09-16T11:44:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330999999999999e-05 2024-09-16T11:44:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442977+0000", 2024-09-16T11:44:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7874000000000003e-05 2024-09-16T11:44:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41956 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.656139893999999, 2024-09-16T11:44:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028766896, 2024-09-16T11:44:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41956 2024-09-16T11:44:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:23.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414326+0000", 2024-09-16T11:44:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9900000000000009e-07 2024-09-16T11:44:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414330+0000", 2024-09-16T11:44:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2339999999999996e-06 2024-09-16T11:44:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414341+0000", 2024-09-16T11:44:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1358999999999999e-05 2024-09-16T11:44:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414347+0000", 2024-09-16T11:44:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8100000000000003e-06 2024-09-16T11:44:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443002+0000", 2024-09-16T11:44:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654331000000002 2024-09-16T11:44:23.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443040+0000", 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8262000000000001e-05 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443092+0000", 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1901000000000003e-05 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41957 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.656112475, 2024-09-16T11:44:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028848318000000001, 2024-09-16T11:44:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41957 2024-09-16T11:44:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414353+0000", 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:23.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414355+0000", 2024-09-16T11:44:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3489999999999999e-06 2024-09-16T11:44:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414364+0000", 2024-09-16T11:44:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1449999999999997e-06 2024-09-16T11:44:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414368+0000", 2024-09-16T11:44:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8940000000000003e-06 2024-09-16T11:44:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443115+0000", 2024-09-16T11:44:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028746532000000002 2024-09-16T11:44:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443152+0000", 2024-09-16T11:44:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6814999999999999e-05 2024-09-16T11:44:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443201+0000", 2024-09-16T11:44:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9069000000000001e-05 2024-09-16T11:44:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41958 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.656088693000001, 2024-09-16T11:44:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028939856, 2024-09-16T11:44:23.110 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41958 2024-09-16T11:44:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414377+0000", 2024-09-16T11:44:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4099999999999999e-07 2024-09-16T11:44:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414379+0000", 2024-09-16T11:44:23.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.418e-06 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414385+0000", 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999999e-06 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414389+0000", 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7740000000000002e-06 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443223+0000", 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028833919 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443263+0000", 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0148999999999998e-05 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443316+0000", 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2636000000000002e-05 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41959 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.656071768, 2024-09-16T11:44:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029040309, 2024-09-16T11:44:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41959 2024-09-16T11:44:23.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:44:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414395+0000", 2024-09-16T11:44:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7579999999999999e-06 2024-09-16T11:44:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414401+0000", 2024-09-16T11:44:23.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5330000000000002e-06 2024-09-16T11:44:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414404+0000", 2024-09-16T11:44:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.529e-06 2024-09-16T11:44:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443339+0000", 2024-09-16T11:44:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028934933999999999 2024-09-16T11:44:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443383+0000", 2024-09-16T11:44:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3832000000000003e-05 2024-09-16T11:44:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443433+0000", 2024-09-16T11:44:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0309000000000002e-05 2024-09-16T11:44:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.119 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41960 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.656056933999999, 2024-09-16T11:44:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029138134, 2024-09-16T11:44:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41960 2024-09-16T11:44:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-07 2024-09-16T11:44:23.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414410+0000", 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414417+0000", 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414420+0000", 2024-09-16T11:44:23.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:44:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443457+0000", 2024-09-16T11:44:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029037065000000001 2024-09-16T11:44:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443495+0000", 2024-09-16T11:44:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7555000000000002e-05 2024-09-16T11:44:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443546+0000", 2024-09-16T11:44:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1422000000000003e-05 2024-09-16T11:44:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41961 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.656040953000002, 2024-09-16T11:44:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029230973, 2024-09-16T11:44:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.124 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41961 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4499999999999998e-07 2024-09-16T11:44:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414432+0000", 2024-09-16T11:44:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.413e-06 2024-09-16T11:44:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414439+0000", 2024-09-16T11:44:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4769999999999997e-06 2024-09-16T11:44:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414443+0000", 2024-09-16T11:44:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8859999999999997e-06 2024-09-16T11:44:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443569+0000", 2024-09-16T11:44:23.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029126124 2024-09-16T11:44:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443606+0000", 2024-09-16T11:44:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6588999999999998e-05 2024-09-16T11:44:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443655+0000", 2024-09-16T11:44:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9138999999999998e-05 2024-09-16T11:44:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41962 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.656017985999998, 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029324505000000001, 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41962 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.129 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:44:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414449+0000", 2024-09-16T11:44:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0279999999999999e-06 2024-09-16T11:44:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414456+0000", 2024-09-16T11:44:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3339999999999998e-06 2024-09-16T11:44:23.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414459+0000", 2024-09-16T11:44:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.382e-06 2024-09-16T11:44:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443678+0000", 2024-09-16T11:44:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029218958999999999 2024-09-16T11:44:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443719+0000", 2024-09-16T11:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0722000000000001e-05 2024-09-16T11:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443771+0000", 2024-09-16T11:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2661000000000003e-05 2024-09-16T11:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.133 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41963 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.656002083000001, 2024-09-16T11:44:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029416557999999999, 2024-09-16T11:44:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41963 2024-09-16T11:44:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414465+0000", 2024-09-16T11:44:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:44:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414470+0000", 2024-09-16T11:44:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4750000000000001e-06 2024-09-16T11:44:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414474+0000", 2024-09-16T11:44:23.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3720000000000001e-06 2024-09-16T11:44:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443795+0000", 2024-09-16T11:44:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029320821 2024-09-16T11:44:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443830+0000", 2024-09-16T11:44:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5463000000000002e-05 2024-09-16T11:44:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443879+0000", 2024-09-16T11:44:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9147000000000002e-05 2024-09-16T11:44:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41964 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.655987195000002, 2024-09-16T11:44:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029521202999999999, 2024-09-16T11:44:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.138 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41964 2024-09-16T11:44:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414480+0000", 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414486+0000", 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414489+0000", 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2770000000000001e-06 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443903+0000", 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029414109000000001 2024-09-16T11:44:23.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443944+0000", 2024-09-16T11:44:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0992999999999999e-05 2024-09-16T11:44:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443999+0000", 2024-09-16T11:44:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4632000000000001e-05 2024-09-16T11:44:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41965 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.655972233, 2024-09-16T11:44:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029611293, 2024-09-16T11:44:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41965 2024-09-16T11:44:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.143 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:23.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414495+0000", 2024-09-16T11:44:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.88e-06 2024-09-16T11:44:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414501+0000", 2024-09-16T11:44:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2489999999999998e-06 2024-09-16T11:44:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414504+0000", 2024-09-16T11:44:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3139999999999999e-06 2024-09-16T11:44:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444023+0000", 2024-09-16T11:44:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029518564000000001 2024-09-16T11:44:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444058+0000", 2024-09-16T11:44:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5196e-05 2024-09-16T11:44:23.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444104+0000", 2024-09-16T11:44:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5732999999999997e-05 2024-09-16T11:44:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41966 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.655956026999998, 2024-09-16T11:44:23.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029707678000000001, 2024-09-16T11:44:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41966 2024-09-16T11:44:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414514+0000", 2024-09-16T11:44:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-16T11:44:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414518+0000", 2024-09-16T11:44:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3839999999999999e-06 2024-09-16T11:44:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414521+0000", 2024-09-16T11:44:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0400000000000001e-06 2024-09-16T11:44:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444128+0000", 2024-09-16T11:44:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029606612000000001 2024-09-16T11:44:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444167+0000", 2024-09-16T11:44:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9418000000000001e-05 2024-09-16T11:44:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444216+0000", 2024-09-16T11:44:23.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9354999999999998e-05 2024-09-16T11:44:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41967 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.655940218000001, 2024-09-16T11:44:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029796916, 2024-09-16T11:44:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41967 2024-09-16T11:44:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6699999999999999e-07 2024-09-16T11:44:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414527+0000", 2024-09-16T11:44:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7060000000000001e-06 2024-09-16T11:44:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414533+0000", 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2140000000000001e-06 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414536+0000", 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444239+0000", 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029702698999999999 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444273+0000", 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4229e-05 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444321+0000", 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8477999999999999e-05 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.156 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41968 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.655925321000002, 2024-09-16T11:44:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029902765000000001, 2024-09-16T11:44:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41968 2024-09-16T11:44:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414540+0000", 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414542+0000", 2024-09-16T11:44:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:44:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414547+0000", 2024-09-16T11:44:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4639999999999997e-06 2024-09-16T11:44:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414551+0000", 2024-09-16T11:44:23.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6279999999999998e-06 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444346+0000", 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029794885 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444389+0000", 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3442000000000002e-05 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444442+0000", 2024-09-16T11:44:23.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3173000000000001e-05 2024-09-16T11:44:23.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.161 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.161 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41969 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:23.161 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:23.161 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.655910409000001, 2024-09-16T11:44:23.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029997612999999999, 2024-09-16T11:44:23.161 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.161 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:23.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41969 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:23.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414555+0000", 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414556+0000", 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.725e-06 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414563+0000", 2024-09-16T11:44:23.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2530000000000001e-06 2024-09-16T11:44:23.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414566+0000", 2024-09-16T11:44:23.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4429999999999999e-06 2024-09-16T11:44:23.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444463+0000", 2024-09-16T11:44:23.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029897271 2024-09-16T11:44:23.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444503+0000", 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9782e-05 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444552+0000", 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8789000000000003e-05 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.165 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:23.166 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-16T11:44:23.167 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:23.168 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:23.164+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:23.265 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:23.265 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:23.265 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:23.265 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:23.267 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:23.265+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:23.273 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:23.289 INFO:teuthology.orchestra.run.smithi070.stdout:5224 writesame 0xb7ae60b thru 0xb7b9000 data_size 0x5f(0xa9f6 bytes) 2024-09-16T11:44:23.306 INFO:teuthology.orchestra.run.smithi070.stdout:5226 write 0xdf0d318 thru 0xdf154b6 (0x819f bytes) 2024-09-16T11:44:23.310 INFO:teuthology.orchestra.run.smithi070.stdout:5227 punch from 0x24857e to 0x24e209, (0x5c8b bytes) 2024-09-16T11:44:23.310 INFO:teuthology.orchestra.run.smithi070.stdout:5229 writesame 0x5eee77c thru 0x5efd215 data_size 0x1(0xea9a bytes) 2024-09-16T11:44:23.346 INFO:teuthology.orchestra.run.smithi070.stdout:5230 trunc from 0xdf154b7 to 0x915cba4 2024-09-16T11:44:23.368 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:23.366+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:23.375 INFO:teuthology.orchestra.run.smithi070.stdout:5231 write 0xe0ad6 thru 0xe481b (0x3d46 bytes) 2024-09-16T11:44:23.376 INFO:teuthology.orchestra.run.smithi070.stdout:5233 read 0x61986f9 thru 0x61a6436 (0xdd3e bytes) 2024-09-16T11:44:23.377 INFO:teuthology.orchestra.run.smithi070.stdout:5234 writesame 0x4830011 thru 0x4830ab7 data_size 0x9(0xaa7 bytes) 2024-09-16T11:44:23.384 INFO:teuthology.orchestra.run.smithi070.stdout:5236 write 0xa6871bb thru 0xa691d1f (0xab65 bytes) 2024-09-16T11:44:23.386 INFO:teuthology.orchestra.run.smithi070.stdout:5237 write 0x81fb411 thru 0x8206329 (0xaf19 bytes) 2024-09-16T11:44:23.388 INFO:teuthology.orchestra.run.smithi070.stdout:5239 write 0x58fd436 thru 0x5908a19 (0xb5e4 bytes) 2024-09-16T11:44:23.389 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:23.389 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:23.389 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:23.389 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:23.390 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:23.390 INFO:teuthology.orchestra.run.smithi070.stdout:5240 punch from 0x862dd20 to 0x8632fef, (0x52cf bytes) 2024-09-16T11:44:23.390 INFO:teuthology.orchestra.run.smithi070.stdout:5242 writesame 0x92f4857 thru 0x92f8900 data_size 0x1f(0x40aa bytes) 2024-09-16T11:44:23.391 INFO:teuthology.orchestra.run.smithi070.stdout:5243 flatten 2024-09-16T11:44:23.396 INFO:teuthology.orchestra.run.smithi070.stdout:5245 punch from 0x810b516 to 0x8113827, (0x8311 bytes) 2024-09-16T11:44:23.396 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:23.399 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:23.404 INFO:teuthology.orchestra.run.smithi070.stdout:5247 trunc from 0xa691d20 to 0x4343bf5 2024-09-16T11:44:23.411 INFO:teuthology.orchestra.run.smithi070.stdout:5248 write 0x1c636aa thru 0x1c66dce (0x3725 bytes) 2024-09-16T11:44:23.412 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:23.428 INFO:teuthology.orchestra.run.smithi070.stdout:5251 write 0xc37e2d1 thru 0xc387d47 (0x9a77 bytes) 2024-09-16T11:44:23.429 INFO:teuthology.orchestra.run.smithi070.stdout:5252 read 0x260146b thru 0x260a7eb (0x9381 bytes) 2024-09-16T11:44:23.435 INFO:teuthology.orchestra.run.smithi070.stdout:5253 trunc from 0xc387d48 to 0x6d9f46f 2024-09-16T11:44:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:5254 write 0xcc5c775 thru 0xcc67893 (0xb11f bytes) 2024-09-16T11:44:23.466 INFO:teuthology.orchestra.run.smithi070.stdout:5255 trunc from 0xcc67894 to 0x4aac21e 2024-09-16T11:44:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:5256 read 0x39b11c5 thru 0x39bf256 (0xe092 bytes) 2024-09-16T11:44:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:5262 clone 34 order 24 su 65536 sc 3 2024-09-16T11:44:23.483 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:23.481+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:23.540 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:23.540 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 70.617710157999994, 2024-09-16T11:44:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:44:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:23.541 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:44:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:23.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:44:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:44:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:44:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:44:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:44:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:44:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:44:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:44:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:44:23.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:44:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 70.617664465999994, 2024-09-16T11:44:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:44:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:44:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:44:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:44:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:44:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:44:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:44:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:44:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:44:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:44:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:44:23.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:44:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:44:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:44:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:44:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:44:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:44:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:44:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:44:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:44:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:44:23.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:44:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:44:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:44:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:23.553 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 70.617607535999994, 2024-09-16T11:44:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:44:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:44:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:44:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:44:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:44:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:44:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:44:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:44:23.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:44:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:44:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:44:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:44:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:44:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:44:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:44:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:44:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:44:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:44:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:44:23.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.559 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 70.617582167999998, 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:44:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:44:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:44:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:44:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:44:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:44:23.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:44:23.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:23.565 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 70.617530200000004, 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.570 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 70.617449089999994, 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:44:23.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:44:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:44:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:44:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:44:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:44:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:44:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:44:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:44:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:44:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:44:23.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:23.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:44:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:44:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:44:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:44:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:44:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:44:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.576 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 70.617423111999997, 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:44:23.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:44:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:44:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:44:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:44:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:44:23.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:44:23.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 70.199904486999998, 2024-09-16T11:44:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:44:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:44:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:44:23.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:44:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:44:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:44:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:44:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:44:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:44:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:44:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:44:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:44:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:44:23.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 70.199882556000006, 2024-09-16T11:44:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:44:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:44:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:44:23.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:44:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:44:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:44:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:44:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:44:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:44:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:44:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:44:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:44:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:44:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:44:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:44:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:44:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:44:23.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:44:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:44:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:44:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:44:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:44:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.594 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 70.199805667999996, 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:44:23.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:44:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:44:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:44:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:44:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:44:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:44:23.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:44:23.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28187 4.17 e52/28 4:ee916bd9:::rbd_data.11c1ce740b46.0000000000000000:head v 52'195, mlcod=52'193)", 2024-09-16T11:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.028386282, 2024-09-16T11:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075854429999999999, 2024-09-16T11:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509349+0000", 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2300000000000001e-06 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513315+0000", 2024-09-16T11:44:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039660590000000001 2024-09-16T11:44:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513328+0000", 2024-09-16T11:44:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.308e-05 2024-09-16T11:44:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513334+0000", 2024-09-16T11:44:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513385+0000", 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1347000000000002e-05 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513407+0000", 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1852999999999999e-05 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516882+0000", 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034749690000000001 2024-09-16T11:44:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516933+0000", 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0952999999999998e-05 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28959 4.8 e53/28 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head v 53'408, mlcod=53'406)", 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.506237226, 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088740420000000004, 2024-09-16T11:44:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031498+0000", 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.652e-06 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033515+0000", 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020165930000000001 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033539+0000", 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4295999999999999e-05 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033548+0000", 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1239999999999994e-06 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033610+0000", 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1371999999999994e-05 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033637+0000", 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7098999999999999e-05 2024-09-16T11:44:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040341+0000", 2024-09-16T11:44:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067045819999999997 2024-09-16T11:44:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040371+0000", 2024-09-16T11:44:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9323999999999999e-05 2024-09-16T11:44:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32303 4.8 e53/28 4:1436ac82:::rbd_data.11c1cab081b3.0000000000000022:head v 53'417, mlcod=53'415)", 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.035811251999998, 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085565160000000001, 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-09-16T11:44:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501930+0000", 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7649999999999996e-06 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501946+0000", 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5301000000000001e-05 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501952+0000", 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502100+0000", 2024-09-16T11:44:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014847600000000001 2024-09-16T11:44:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502120+0000", 2024-09-16T11:44:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9303000000000001e-05 2024-09-16T11:44:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510436+0000", 2024-09-16T11:44:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083166420000000008 2024-09-16T11:44:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510479+0000", 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2979000000000003e-05 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32306 4.1e e53/28 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head v 53'124, mlcod=53'122)", 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.035745257999999, 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084990900000000008, 2024-09-16T11:44:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501994+0000", 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502002+0000", 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3839999999999997e-06 2024-09-16T11:44:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502007+0000", 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6290000000000001e-06 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502454+0000", 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00044736800000000002 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502471+0000", 2024-09-16T11:44:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6648999999999999e-05 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510455+0000", 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0079841909999999999 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510488+0000", 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2948999999999999e-05 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32307 4.8 e53/28 4:11c1c37d:::rbd_data.11c1cab081b3.0000000000000026:head v 53'418, mlcod=53'415)", 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 22.034989923000001, 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00776212, 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502750+0000", 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:44:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502760+0000", 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502767+0000", 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4219999999999997e-06 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502802+0000", 2024-09-16T11:44:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5089999999999998e-05 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502816+0000", 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4092e-05 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510483+0000", 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00766704 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510506+0000", 2024-09-16T11:44:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3195999999999999e-05 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33583 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'416, mlcod=53'411)", 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 21.327031514000002, 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078132459999999994, 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210703+0000", 2024-09-16T11:44:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210706+0000", 2024-09-16T11:44:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-06 2024-09-16T11:44:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210712+0000", 2024-09-16T11:44:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:44:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210715+0000", 2024-09-16T11:44:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210736+0000", 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0293999999999999e-05 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210746+0000", 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0178e-05 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218477+0000", 2024-09-16T11:44:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007731501 2024-09-16T11:44:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218516+0000", 2024-09-16T11:44:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8148000000000001e-05 2024-09-16T11:44:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33584 4.0 e53/28 4:044e4d16:::rbd_data.11c1cab081b3.0000000000000019:head v 53'176, mlcod=53'174)", 2024-09-16T11:44:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 21.326220030999998, 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070059470000000002, 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211514+0000", 2024-09-16T11:44:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.37e-07 2024-09-16T11:44:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211524+0000", 2024-09-16T11:44:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4830000000000005e-06 2024-09-16T11:44:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211535+0000", 2024-09-16T11:44:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0842000000000001e-05 2024-09-16T11:44:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211539+0000", 2024-09-16T11:44:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9740000000000004e-06 2024-09-16T11:44:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211584+0000", 2024-09-16T11:44:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5655000000000002e-05 2024-09-16T11:44:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.211602+0000", 2024-09-16T11:44:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7756000000000001e-05 2024-09-16T11:44:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218497+0000", 2024-09-16T11:44:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0068944590000000004 2024-09-16T11:44:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218520+0000", 2024-09-16T11:44:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3141000000000001e-05 2024-09-16T11:44:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33590 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'419, mlcod=53'415)", 2024-09-16T11:44:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 21.315973372999999, 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082499879999999998, 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221761+0000", 2024-09-16T11:44:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-16T11:44:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221765+0000", 2024-09-16T11:44:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1590000000000003e-06 2024-09-16T11:44:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221774+0000", 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7490000000000004e-06 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221778+0000", 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0419999999999996e-06 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221800+0000", 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1926999999999999e-05 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221810+0000", 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0079e-05 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.229980+0000", 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081698469999999992 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230010+0000", 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0746e-05 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33591 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'420, mlcod=53'415)", 2024-09-16T11:44:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 21.315411889, 2024-09-16T11:44:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007702576, 2024-09-16T11:44:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222326+0000", 2024-09-16T11:44:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:44:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222337+0000", 2024-09-16T11:44:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1008e-05 2024-09-16T11:44:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222342+0000", 2024-09-16T11:44:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0050000000000004e-06 2024-09-16T11:44:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222366+0000", 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3696000000000001e-05 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222385+0000", 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230012+0000", 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076270940000000001 2024-09-16T11:44:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230025+0000", 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2112e-05 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33593 4.1e e53/28 4:7e77cec2:::rbd_data.11c1cab081b3.000000000000000c:head v 53'133, mlcod=53'131)", 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 21.310777967, 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070516270000000004, 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226957+0000", 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227014+0000", 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7074999999999998e-05 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227024+0000", 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0617e-05 2024-09-16T11:44:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227030+0000", 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3279999999999996e-06 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227061+0000", 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1718999999999999e-05 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227074+0000", 2024-09-16T11:44:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2738000000000001e-05 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.233975+0000", 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0069008480000000002 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.234008+0000", 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2614000000000003e-05 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:23.648 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:23.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:23.650 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:23.647+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:23.704 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:44:23.704 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:44:23.704 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T11:44:23.704 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T11:44:23.750 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:23.750 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:23.750 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:23.750 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:23.751 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:23.749+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:23.759 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:23.852 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:23.850+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:23.859 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:23.859 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:23.860 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:23.860 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:23.860 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:23.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:23.953 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:23.950+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:23.971 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:23.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:23.972 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:23.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.972 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:9081 4.b 4:d317bc5b:::rbd_data.11c1dc795e15.000000000000000d:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:44:23.972 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:44:23.972 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.914295627999998, 2024-09-16T11:44:23.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027796406999999999, 2024-09-16T11:44:23.972 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:23.972 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:23.972 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:23.972 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:23.972 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.972 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 9081 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:23.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053957+0000", 2024-09-16T11:44:23.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7010000000000001e-06 2024-09-16T11:44:23.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:23.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053965+0000", 2024-09-16T11:44:23.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5939999999999996e-06 2024-09-16T11:44:23.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:23.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053980+0000", 2024-09-16T11:44:23.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5099000000000001e-05 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053986+0000", 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5759999999999996e-06 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.054023+0000", 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999999e-05 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:23.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.054076+0000", 2024-09-16T11:44:23.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2852999999999997e-05 2024-09-16T11:44:23.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:44:23.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.058880+0000", 2024-09-16T11:44:23.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048042639999999999 2024-09-16T11:44:23.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:23.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081663+0000", 2024-09-16T11:44:23.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022782908000000001 2024-09-16T11:44:23.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081672+0000", 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0440000000000001e-06 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081707+0000", 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4986e-05 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081752+0000", 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4781999999999999e-05 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:23.977 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:45776 4.1e 4:7e0160b4:::rbd_data.11c1e170f5f0.000000000000000d:1b [read 458752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.288850+0000", 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.679401404, 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022448177999999999, 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 45776 2024-09-16T11:44:23.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.979 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:23.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:23.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.288850+0000", 2024-09-16T11:44:23.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:23.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:23.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.288850+0000", 2024-09-16T11:44:23.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:23.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.288850+0000", 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4700000000000002e-07 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.288852+0000", 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0040000000000002e-06 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.288858+0000", 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:44:23.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.288861+0000", 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2799999999999999e-06 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.311175+0000", 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022313432000000001 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.311242+0000", 2024-09-16T11:44:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7112000000000004e-05 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.311298+0000", 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6328e-05 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:45899 4.1e 4:7c82b124:::rbd_data.11c1e170f5f0.000000000000002c:1b [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.289348+0000", 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.678903302, 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022287114, 2024-09-16T11:44:23.982 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 45899 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289348+0000", 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:23.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289348+0000", 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289348+0000", 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289350+0000", 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9570000000000001e-06 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289356+0000", 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289365+0000", 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0559999999999994e-06 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.311495+0000", 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022129672 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.311564+0000", 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8973999999999994e-05 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.311635+0000", 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1229000000000005e-05 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:45900 4.1e 4:7c82b124:::rbd_data.11c1e170f5f0.000000000000002c:1b [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:23.986 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.289373+0000", 2024-09-16T11:44:23.987 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.678878029, 2024-09-16T11:44:23.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022434593999999999, 2024-09-16T11:44:23.987 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:23.987 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:23.987 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:23.987 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:23.987 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.987 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 45900 2024-09-16T11:44:23.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.987 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:23.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.987 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:23.987 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289373+0000", 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289373+0000", 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289374+0000", 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289376+0000", 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.978e-06 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289382+0000", 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9569999999999999e-06 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289385+0000", 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2490000000000001e-06 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.311658+0000", 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022273718000000001 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.311730+0000", 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1876000000000001e-05 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.311808+0000", 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7423999999999996e-05 2024-09-16T11:44:23.990 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:23.991 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:23.991 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:23.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.991 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:45901 4.1e 4:7c82b124:::rbd_data.11c1e170f5f0.000000000000002c:1b [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:23.991 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.289388+0000", 2024-09-16T11:44:23.991 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.67886283, 2024-09-16T11:44:23.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022600921999999999, 2024-09-16T11:44:23.991 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:23.991 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:23.991 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:23.991 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:23.991 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 45901 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289388+0000", 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289388+0000", 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:23.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289389+0000", 2024-09-16T11:44:23.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:44:23.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:23.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289391+0000", 2024-09-16T11:44:23.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7749999999999999e-06 2024-09-16T11:44:23.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:23.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289396+0000", 2024-09-16T11:44:23.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289400+0000", 2024-09-16T11:44:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6600000000000001e-06 2024-09-16T11:44:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.311833+0000", 2024-09-16T11:44:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022432777000000001 2024-09-16T11:44:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.311904+0000", 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1347999999999995e-05 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.311989+0000", 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5235999999999995e-05 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:45902 4.1e 4:7c82b124:::rbd_data.11c1e170f5f0.000000000000002c:1b [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.289404+0000", 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.678847516999999, 2024-09-16T11:44:23.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022760790999999999, 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 45902 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289404+0000", 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289404+0000", 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289404+0000", 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8999999999999997e-07 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289406+0000", 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289413+0000", 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9789999999999996e-06 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289417+0000", 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8709999999999999e-06 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.312015+0000", 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022598627999999999 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:23.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.312076+0000", 2024-09-16T11:44:23.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0745000000000002e-05 2024-09-16T11:44:23.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:23.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.312165+0000", 2024-09-16T11:44:23.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8320999999999997e-05 2024-09-16T11:44:23.999 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:23.999 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:23.999 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:23.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:23.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:23.999 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:45903 4.1e 4:7c82b124:::rbd_data.11c1e170f5f0.000000000000002c:1b [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.000 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.289430+0000", 2024-09-16T11:44:24.000 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.678820758000001, 2024-09-16T11:44:24.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022867851000000002, 2024-09-16T11:44:24.000 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.000 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.000 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.000 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.000 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.000 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 45903 2024-09-16T11:44:24.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.000 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289430+0000", 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289430+0000", 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289431+0000", 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289433+0000", 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1629999999999999e-06 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289440+0000", 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2690000000000002e-06 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289444+0000", 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6500000000000002e-06 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.312190+0000", 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022745726000000001 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.312250+0000", 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0100999999999999e-05 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.312298+0000", 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8448000000000001e-05 2024-09-16T11:44:24.003 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.004 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.004 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.004 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:45904 4.1e 4:7c82b124:::rbd_data.11c1e170f5f0.000000000000002c:1b [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.004 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.289448+0000", 2024-09-16T11:44:24.004 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.678803584000001, 2024-09-16T11:44:24.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022994415000000001, 2024-09-16T11:44:24.004 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.004 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.004 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.004 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 45904 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289448+0000", 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289448+0000", 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289448+0000", 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5000000000000003e-07 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289450+0000", 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289457+0000", 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9070000000000001e-06 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289460+0000", 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3699999999999999e-06 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.312320+0000", 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022860051999999999 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.312387+0000", 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6353999999999995e-05 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.312442+0000", 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5318000000000001e-05 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:45905 4.1e 4.248d413e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.289465+0000", 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.678786141, 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024768528000000001, 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.008 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:24.009 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.009 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.009 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.009 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 45905 2024-09-16T11:44:24.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.009 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289465+0000", 2024-09-16T11:44:24.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289465+0000", 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289466+0000", 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0900000000000002e-07 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289481+0000", 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5291999999999999e-05 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289488+0000", 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8379999999999996e-06 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289491+0000", 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6150000000000001e-06 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.314228+0000", 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024736719000000001 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.012 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.314234+0000", 2024-09-16T11:44:24.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5550000000000002e-06 2024-09-16T11:44:24.012 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.012 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.012 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.012 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:46761 4.6 4:63f1f241:::rbd_data.11c1e170f5f0.000000000000001c:1b [read 385024~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.012 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.326681+0000", 2024-09-16T11:44:24.012 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.641570176, 2024-09-16T11:44:24.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022450253999999999, 2024-09-16T11:44:24.012 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 46761 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326681+0000", 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326681+0000", 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326681+0000", 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0400000000000002e-07 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326683+0000", 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.37e-06 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326688+0000", 2024-09-16T11:44:24.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5060000000000003e-06 2024-09-16T11:44:24.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326691+0000", 2024-09-16T11:44:24.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.134e-06 2024-09-16T11:44:24.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.348792+0000", 2024-09-16T11:44:24.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022100754 2024-09-16T11:44:24.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.348883+0000", 2024-09-16T11:44:24.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0890000000000006e-05 2024-09-16T11:44:24.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349131+0000", 2024-09-16T11:44:24.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024829600000000001 2024-09-16T11:44:24.017 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.017 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.017 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.017 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:46762 4.6 4:63f1f241:::rbd_data.11c1e170f5f0.000000000000001c:1b [read 475136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.017 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.326694+0000", 2024-09-16T11:44:24.017 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.641556982999999, 2024-09-16T11:44:24.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022589603999999999, 2024-09-16T11:44:24.017 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.017 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.017 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.018 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.018 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.018 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 46762 2024-09-16T11:44:24.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.018 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.018 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.018 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326694+0000", 2024-09-16T11:44:24.018 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.018 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.018 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326694+0000", 2024-09-16T11:44:24.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326695+0000", 2024-09-16T11:44:24.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8799999999999998e-07 2024-09-16T11:44:24.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326701+0000", 2024-09-16T11:44:24.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8600000000000004e-06 2024-09-16T11:44:24.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326707+0000", 2024-09-16T11:44:24.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6740000000000002e-06 2024-09-16T11:44:24.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326710+0000", 2024-09-16T11:44:24.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.349e-06 2024-09-16T11:44:24.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349167+0000", 2024-09-16T11:44:24.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022456598000000001 2024-09-16T11:44:24.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.021 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.021 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349214+0000", 2024-09-16T11:44:24.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6990000000000002e-05 2024-09-16T11:44:24.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.021 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.021 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349284+0000", 2024-09-16T11:44:24.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9845000000000006e-05 2024-09-16T11:44:24.021 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.021 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:46763 4.6 4:63f1f241:::rbd_data.11c1e170f5f0.000000000000001c:1b [read 516096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.326714+0000", 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.641537291000001, 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022776635999999999, 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 46763 2024-09-16T11:44:24.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326714+0000", 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326714+0000", 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326714+0000", 2024-09-16T11:44:24.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:44:24.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326716+0000", 2024-09-16T11:44:24.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5600000000000001e-06 2024-09-16T11:44:24.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326721+0000", 2024-09-16T11:44:24.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5079999999999996e-06 2024-09-16T11:44:24.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326724+0000", 2024-09-16T11:44:24.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1769999999999998e-06 2024-09-16T11:44:24.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349330+0000", 2024-09-16T11:44:24.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022605935000000001 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349418+0000", 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7674000000000001e-05 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349491+0000", 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2482000000000002e-05 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:46788 4.16 4:6ec64c8a:::rbd_data.11c1e521eb15.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 8060928] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.326728+0000", 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.641523319999999, 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028859883999999999, 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 46788 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326728+0000", 2024-09-16T11:44:24.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326728+0000", 2024-09-16T11:44:24.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326728+0000", 2024-09-16T11:44:24.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:24.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327570+0000", 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00084174300000000005 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327582+0000", 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2435e-05 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327589+0000", 2024-09-16T11:44:24.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1970000000000001e-06 2024-09-16T11:44:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349312+0000", 2024-09-16T11:44:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.021723524000000001 2024-09-16T11:44:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349465+0000", 2024-09-16T11:44:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015305200000000001 2024-09-16T11:44:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3", 2024-09-16T11:44:24.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350219+0000", 2024-09-16T11:44:24.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00075365399999999998 2024-09-16T11:44:24.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:24.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.355296+0000", 2024-09-16T11:44:24.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0050773210000000001 2024-09-16T11:44:24.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:24.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.355469+0000", 2024-09-16T11:44:24.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017311400000000001 2024-09-16T11:44:24.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.355552+0000", 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3018999999999999e-05 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.355588+0000", 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5468e-05 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:46791 4.1e 4:7c82b124:::rbd_data.11c1e170f5f0.000000000000002c:1b [read 122880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.327629+0000", 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.640622694999999, 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022220539000000001, 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 46791 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.033 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327629+0000", 2024-09-16T11:44:24.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327629+0000", 2024-09-16T11:44:24.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327629+0000", 2024-09-16T11:44:24.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7400000000000003e-07 2024-09-16T11:44:24.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327631+0000", 2024-09-16T11:44:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.103e-06 2024-09-16T11:44:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327637+0000", 2024-09-16T11:44:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8810000000000001e-06 2024-09-16T11:44:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327640+0000", 2024-09-16T11:44:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.365e-06 2024-09-16T11:44:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349701+0000", 2024-09-16T11:44:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022060842000000001 2024-09-16T11:44:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349771+0000", 2024-09-16T11:44:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0084000000000002e-05 2024-09-16T11:44:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349849+0000", 2024-09-16T11:44:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7689999999999996e-05 2024-09-16T11:44:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:46792 4.1e 4:7c82b124:::rbd_data.11c1e170f5f0.000000000000002c:1b [read 163840~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.327644+0000", 2024-09-16T11:44:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.640607080000001, 2024-09-16T11:44:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022375401, 2024-09-16T11:44:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 46792 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327644+0000", 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327644+0000", 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327645+0000", 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327646+0000", 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7749999999999999e-06 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327652+0000", 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7579999999999998e-06 2024-09-16T11:44:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327656+0000", 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349871+0000", 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022215123 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349942+0000", 2024-09-16T11:44:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1511e-05 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350020+0000", 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7406999999999999e-05 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:46793 4.1e 4:7c82b124:::rbd_data.11c1e170f5f0.000000000000002c:1b [read 253952~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.327659+0000", 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.640592082, 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022510829999999999, 2024-09-16T11:44:24.042 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.043 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.043 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.043 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.043 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.043 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 46793 2024-09-16T11:44:24.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.043 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327659+0000", 2024-09-16T11:44:24.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327659+0000", 2024-09-16T11:44:24.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327660+0000", 2024-09-16T11:44:24.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:44:24.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.045 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327661+0000", 2024-09-16T11:44:24.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.731e-06 2024-09-16T11:44:24.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.045 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.045 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327667+0000", 2024-09-16T11:44:24.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.524e-06 2024-09-16T11:44:24.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.045 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.045 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327676+0000", 2024-09-16T11:44:24.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1279999999999997e-06 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350039+0000", 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022363047 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350105+0000", 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5821000000000006e-05 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350170+0000", 2024-09-16T11:44:24.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5179999999999996e-05 2024-09-16T11:44:24.047 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.047 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.047 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.047 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:46794 4.1e 4:7c82b124:::rbd_data.11c1e170f5f0.000000000000002c:1b [read 294912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.047 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.327680+0000", 2024-09-16T11:44:24.047 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.640571409, 2024-09-16T11:44:24.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022635389999999998, 2024-09-16T11:44:24.047 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.047 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.047 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.048 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.048 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.048 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 46794 2024-09-16T11:44:24.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.048 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327680+0000", 2024-09-16T11:44:24.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327680+0000", 2024-09-16T11:44:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327680+0000", 2024-09-16T11:44:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.77e-07 2024-09-16T11:44:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327682+0000", 2024-09-16T11:44:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9520000000000001e-06 2024-09-16T11:44:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327688+0000", 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0000000000000002e-06 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327692+0000", 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.388e-06 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350187+0000", 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022495953999999999 2024-09-16T11:44:24.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350248+0000", 2024-09-16T11:44:24.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0739000000000002e-05 2024-09-16T11:44:24.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350315+0000", 2024-09-16T11:44:24.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6979999999999999e-05 2024-09-16T11:44:24.051 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.051 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.051 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:46795 4.1e 4:7c82b124:::rbd_data.11c1e170f5f0.000000000000002c:1b [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.327695+0000", 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.640556102, 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022756980999999999, 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 46795 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.052 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327695+0000", 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327695+0000", 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327696+0000", 2024-09-16T11:44:24.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7e-07 2024-09-16T11:44:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327697+0000", 2024-09-16T11:44:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.652e-06 2024-09-16T11:44:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327703+0000", 2024-09-16T11:44:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.716e-06 2024-09-16T11:44:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327706+0000", 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.286e-06 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350333+0000", 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022627110999999998 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350398+0000", 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4548999999999995e-05 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350452+0000", 2024-09-16T11:44:24.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4296999999999999e-05 2024-09-16T11:44:24.056 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.056 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.056 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.056 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:46796 4.1e 4:7c82b124:::rbd_data.11c1e170f5f0.000000000000002c:1b [read 425984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.056 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.327710+0000", 2024-09-16T11:44:24.056 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.640541647999999, 2024-09-16T11:44:24.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022884778000000001, 2024-09-16T11:44:24.056 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.057 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.057 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.057 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.057 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.057 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 46796 2024-09-16T11:44:24.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.057 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327710+0000", 2024-09-16T11:44:24.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327710+0000", 2024-09-16T11:44:24.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327710+0000", 2024-09-16T11:44:24.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:44:24.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327711+0000", 2024-09-16T11:44:24.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5799999999999999e-06 2024-09-16T11:44:24.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327717+0000", 2024-09-16T11:44:24.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1850000000000003e-06 2024-09-16T11:44:24.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327720+0000", 2024-09-16T11:44:24.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.123e-06 2024-09-16T11:44:24.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350472+0000", 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022752715 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350541+0000", 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8048999999999998e-05 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350594+0000", 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3817000000000003e-05 2024-09-16T11:44:24.060 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:46797 4.1e 4:7c82b124:::rbd_data.11c1e170f5f0.000000000000002c:1b [read 516096~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.327723+0000", 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.640527795000001, 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.023007811, 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.061 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 46797 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327723+0000", 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327723+0000", 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327724+0000", 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9400000000000001e-07 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327726+0000", 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.711e-06 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327731+0000", 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5969999999999999e-06 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327735+0000", 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4620000000000001e-06 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350613+0000", 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022878234000000001 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350678+0000", 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5398e-05 2024-09-16T11:44:24.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350731+0000", 2024-09-16T11:44:24.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3015e-05 2024-09-16T11:44:24.065 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.065 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.065 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.065 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.065 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.065 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:24.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:44:24.067 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:24.064+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:24.067 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:24.067 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:24.067 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 73, 2024-09-16T11:44:24.067 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "3420bcad-625f-4e15-aaff-fb4236322168", 2024-09-16T11:44:24.068 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2024-09-16T11:43:01.654626+0000", 2024-09-16T11:44:24.068 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2024-09-16T11:44:23.963139+0000", 2024-09-16T11:44:24.068 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2024-09-16T11:43:11.689589+0000", 2024-09-16T11:44:24.068 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2024-09-16T11:43:04.526236+0000", 2024-09-16T11:44:24.068 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T11:44:24.068 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5806080, 2024-09-16T11:44:24.068 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2024-09-16T11:44:24.068 INFO:teuthology.orchestra.run.smithi070.stdout: "nodeep-scrub", 2024-09-16T11:44:24.068 INFO:teuthology.orchestra.run.smithi070.stdout: "noscrub", 2024-09-16T11:44:24.068 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2024-09-16T11:44:24.068 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2024-09-16T11:44:24.068 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2024-09-16T11:44:24.068 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2024-09-16T11:44:24.069 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:24.069 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 5, 2024-09-16T11:44:24.069 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T11:44:24.069 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T11:44:24.069 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T11:44:24.069 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2024-09-16T11:44:24.069 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 4, 2024-09-16T11:44:24.069 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2024-09-16T11:44:24.069 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2024-09-16T11:44:24.069 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2024-09-16T11:44:24.069 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "quincy", 2024-09-16T11:44:24.069 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2024-09-16T11:44:24.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.070 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2024-09-16T11:44:24.070 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2024-09-16T11:44:24.070 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T11:43:11.406582+0000", 2024-09-16T11:44:24.070 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T11:44:24.070 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T11:44:24.070 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T11:44:24.070 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-16T11:44:24.070 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-16T11:44:24.070 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T11:44:24.070 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:44:24.070 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:44:24.070 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:44:24.070 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:44:24.071 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T11:44:24.071 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:44:24.071 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2024-09-16T11:44:24.071 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2024-09-16T11:44:24.071 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2024-09-16T11:44:24.071 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2024-09-16T11:44:24.071 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2024-09-16T11:44:24.071 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T11:44:24.071 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T11:44:24.071 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T11:44:24.071 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T11:44:24.071 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T11:44:24.071 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T11:44:24.072 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T11:44:24.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.072 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "16", 2024-09-16T11:44:24.072 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T11:44:24.072 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:44:24.072 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:44:24.072 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T11:44:24.072 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:44:24.072 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T11:44:24.072 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T11:44:24.072 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T11:44:24.072 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T11:44:24.072 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T11:44:24.073 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T11:44:24.073 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T11:44:24.073 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T11:44:24.073 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T11:44:24.073 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T11:44:24.073 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T11:44:24.073 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T11:44:24.073 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T11:44:24.073 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:44:24.073 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:44:24.073 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:44:24.073 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T11:44:24.073 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T11:44:24.074 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T11:44:24.074 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T11:44:24.074 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T11:44:24.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.074 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T11:44:24.074 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T11:44:24.074 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T11:44:24.074 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:44:24.074 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:44:24.074 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:44:24.074 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:44:24.074 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2024-09-16T11:44:24.075 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2024-09-16T11:44:24.076 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T11:43:13.125709+0000", 2024-09-16T11:44:24.076 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T11:44:24.076 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T11:44:24.076 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T11:44:24.076 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-16T11:44:24.076 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-16T11:44:24.076 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T11:44:24.076 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:44:24.076 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:44:24.076 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:44:24.076 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:44:24.076 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T11:44:24.076 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:44:24.077 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2024-09-16T11:44:24.077 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2024-09-16T11:44:24.077 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2024-09-16T11:44:24.077 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2024-09-16T11:44:24.077 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2024-09-16T11:44:24.077 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T11:44:24.077 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T11:44:24.077 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T11:44:24.077 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T11:44:24.077 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T11:44:24.077 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T11:44:24.077 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T11:44:24.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.078 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "23", 2024-09-16T11:44:24.078 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "23", 2024-09-16T11:44:24.078 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "23", 2024-09-16T11:44:24.078 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "23", 2024-09-16T11:44:24.078 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T11:44:24.078 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:44:24.078 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 3, 2024-09-16T11:44:24.078 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 21, 2024-09-16T11:44:24.078 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T11:44:24.078 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T11:44:24.078 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T11:44:24.078 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T11:44:24.079 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [ 2024-09-16T11:44:24.079 INFO:teuthology.orchestra.run.smithi070.stdout: 3 2024-09-16T11:44:24.079 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:24.079 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T11:44:24.079 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": 3, 2024-09-16T11:44:24.079 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": 3, 2024-09-16T11:44:24.079 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T11:44:24.079 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T11:44:24.079 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T11:44:24.079 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:44:24.079 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:44:24.079 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:44:24.079 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T11:44:24.080 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T11:44:24.080 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T11:44:24.080 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T11:44:24.080 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T11:44:24.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.080 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T11:44:24.080 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T11:44:24.080 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T11:44:24.080 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:44:24.080 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:44:24.080 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:44:24.080 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:44:24.080 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T11:44:24.081 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T11:44:24.081 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T11:44:24.081 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T11:44:24.081 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T11:44:24.081 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T11:44:24.081 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T11:44:24.081 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.081 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 3, 2024-09-16T11:44:24.081 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cache", 2024-09-16T11:44:24.081 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T11:43:21.227888+0000", 2024-09-16T11:44:24.082 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8201, 2024-09-16T11:44:24.082 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,incomplete_clones,selfmanaged_snaps", 2024-09-16T11:44:24.082 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T11:44:24.082 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-16T11:44:24.082 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-16T11:44:24.082 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T11:44:24.082 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:44:24.082 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:44:24.082 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:44:24.082 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:44:24.082 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T11:44:24.082 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:44:24.082 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 4, 2024-09-16T11:44:24.083 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 4, 2024-09-16T11:44:24.083 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 4, 2024-09-16T11:44:24.083 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 4, 2024-09-16T11:44:24.083 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 4, 2024-09-16T11:44:24.083 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T11:44:24.083 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T11:44:24.083 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T11:44:24.083 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T11:44:24.083 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T11:44:24.083 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T11:44:24.083 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T11:44:24.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.083 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "27", 2024-09-16T11:44:24.084 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "23", 2024-09-16T11:44:24.084 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "23", 2024-09-16T11:44:24.084 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "23", 2024-09-16T11:44:24.084 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T11:44:24.084 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:44:24.084 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 3, 2024-09-16T11:44:24.084 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 21, 2024-09-16T11:44:24.084 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T11:44:24.084 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T11:44:24.084 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T11:44:24.084 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T11:44:24.084 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T11:44:24.084 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": 2, 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "writeback", 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 250, 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bloom", 2024-09-16T11:44:24.085 INFO:teuthology.orchestra.run.smithi070.stdout: "false_positive_probability": 0.050000000000000003, 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size": 0, 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: "seed": 0 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 60, 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 8, 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [ 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:24.086 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 4, 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "pool_client.0", 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T11:43:30.105533+0000", 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:44:24.088 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T11:44:24.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "73", 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 37, 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 73, 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T11:44:24.090 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T11:44:24.091 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T11:44:24.091 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T11:44:24.091 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T11:44:24.091 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T11:44:24.091 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T11:44:24.091 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:44:24.091 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:44:24.091 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:44:24.091 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T11:44:24.091 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T11:44:24.091 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T11:44:24.091 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T11:44:24.091 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T11:44:24.092 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T11:44:24.093 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T11:44:24.093 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.093 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.093 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:24.093 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T11:44:24.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.093 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T11:44:24.093 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "91f052e4-4c8f-4484-92ce-bd1a7f8f3efb", 2024-09-16T11:44:24.093 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:24.093 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:24.093 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:24.093 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.096 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.096 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2024-09-16T11:44:24.096 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:24.096 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.096 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.096 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:24.096 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.096 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.096 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2024-09-16T11:44:24.096 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:24.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2024-09-16T11:44:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2024-09-16T11:44:24.097 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2024-09-16T11:44:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/33355", 2024-09-16T11:44:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/33355", 2024-09-16T11:44:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/33355", 2024-09-16T11:44:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/33355", 2024-09-16T11:44:24.098 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:24.099 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:24.099 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:24.099 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.099 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T11:44:24.099 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "568b1280-7365-4094-afe8-b0b5e766de6f", 2024-09-16T11:44:24.099 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:24.099 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:24.099 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:24.099 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:24.099 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:24.099 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:24.100 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 14, 2024-09-16T11:44:24.100 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 61, 2024-09-16T11:44:24.100 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:24.100 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:24.100 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:24.100 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.100 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.100 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2024-09-16T11:44:24.100 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:24.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.101 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.101 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2024-09-16T11:44:24.101 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:24.101 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.101 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.101 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:24.101 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.101 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.101 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2024-09-16T11:44:24.101 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:24.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2024-09-16T11:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2024-09-16T11:44:24.102 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2024-09-16T11:44:24.103 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:24.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.104 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.104 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2024-09-16T11:44:24.104 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:24.104 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.104 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.104 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/33359", 2024-09-16T11:44:24.104 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/33359", 2024-09-16T11:44:24.104 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/33359", 2024-09-16T11:44:24.104 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/33359", 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "328adfa3-c1b4-4745-9759-7327c8cbd499", 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:24.105 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:24.106 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T11:44:24.106 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:24.106 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:24.106 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:24.106 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:24.106 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.106 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.106 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2024-09-16T11:44:24.106 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:24.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.107 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.107 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2024-09-16T11:44:24.107 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:24.107 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.107 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.107 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:24.107 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.107 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.107 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.107 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2024-09-16T11:44:24.107 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:24.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.108 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.108 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2024-09-16T11:44:24.108 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:24.108 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.108 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.108 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:24.108 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.108 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.108 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2024-09-16T11:44:24.108 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:24.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.109 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.109 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2024-09-16T11:44:24.109 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:24.109 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.109 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.109 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:24.109 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.109 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.109 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/33356", 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/33356", 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6815/33356", 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/33356", 2024-09-16T11:44:24.110 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:24.111 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:24.111 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:24.111 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.111 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T11:44:24.111 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "6df66ae7-e835-4d8f-bd27-243d7a9705b5", 2024-09-16T11:44:24.111 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:24.111 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:24.111 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:24.111 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:24.111 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:24.111 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:24.112 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 14, 2024-09-16T11:44:24.112 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:24.112 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:24.112 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:24.112 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:24.112 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.112 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.112 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2024-09-16T11:44:24.112 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:24.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.112 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.113 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2024-09-16T11:44:24.113 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:24.113 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.113 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.113 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:24.113 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.113 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.113 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2024-09-16T11:44:24.113 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:24.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.114 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.114 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2024-09-16T11:44:24.114 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:24.114 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.114 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.114 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:24.114 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.114 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.114 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2024-09-16T11:44:24.114 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:24.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/33384", 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/33384", 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/33384", 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/33384", 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.117 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.117 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T11:44:24.117 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "29f52d62-d4c3-49f9-992f-23817f5622d2", 2024-09-16T11:44:24.117 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:24.117 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:24.117 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:24.117 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:24.117 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:24.117 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:24.117 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T11:44:24.117 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:24.117 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:24.117 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6800", 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6803", 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6805", 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6807", 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.120 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:24.120 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.120 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.120 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6813", 2024-09-16T11:44:24.120 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:24.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.120 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.120 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6815", 2024-09-16T11:44:24.120 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:24.120 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.120 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.121 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:24.121 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.121 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.121 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6809", 2024-09-16T11:44:24.121 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:24.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.121 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.121 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6811", 2024-09-16T11:44:24.121 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:24.121 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.122 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.122 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.192:6803/33331", 2024-09-16T11:44:24.122 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.192:6807/33331", 2024-09-16T11:44:24.122 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.192:6815/33331", 2024-09-16T11:44:24.122 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.192:6811/33331", 2024-09-16T11:44:24.122 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:24.122 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:24.122 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:24.122 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.122 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.122 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "0c1a275a-3c75-4e0b-bd35-d34b64d99c0a", 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 14, 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 60, 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.124 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.124 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6816", 2024-09-16T11:44:24.124 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:24.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.124 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.124 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.124 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6817", 2024-09-16T11:44:24.124 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:24.124 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.124 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.125 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:24.125 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.125 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.125 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6818", 2024-09-16T11:44:24.125 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:24.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.125 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.125 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6819", 2024-09-16T11:44:24.125 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:24.125 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.125 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.126 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:24.126 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.126 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.126 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.126 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6822", 2024-09-16T11:44:24.126 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:24.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.126 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.126 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.126 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6823", 2024-09-16T11:44:24.126 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:24.126 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.127 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.127 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.127 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:24.127 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.127 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.127 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.127 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6820", 2024-09-16T11:44:24.127 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:24.127 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.127 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.127 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.127 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6821", 2024-09-16T11:44:24.127 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:24.128 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.128 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.128 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.192:6817/33333", 2024-09-16T11:44:24.128 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.192:6819/33333", 2024-09-16T11:44:24.128 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.192:6823/33333", 2024-09-16T11:44:24.128 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.192:6821/33333", 2024-09-16T11:44:24.128 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:24.128 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:24.128 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:24.128 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.129 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T11:44:24.129 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "d79f0070-ce58-48e7-a56a-2762ec76a271", 2024-09-16T11:44:24.129 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:24.129 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:24.129 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:24.129 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:24.129 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:24.129 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:24.129 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T11:44:24.129 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:24.129 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:24.129 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6801", 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6802", 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.131 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:24.131 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.131 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.131 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6804", 2024-09-16T11:44:24.131 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:24.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.131 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.131 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6806", 2024-09-16T11:44:24.131 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:24.131 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.131 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.132 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:24.132 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.132 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.132 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6812", 2024-09-16T11:44:24.132 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:24.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.132 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.132 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6814", 2024-09-16T11:44:24.132 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:24.132 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.133 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.133 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:24.133 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.133 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.133 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6808", 2024-09-16T11:44:24.133 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:24.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.133 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.133 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6810", 2024-09-16T11:44:24.133 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:24.134 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.134 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.134 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.192:6802/33332", 2024-09-16T11:44:24.134 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.192:6806/33332", 2024-09-16T11:44:24.134 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.192:6814/33332", 2024-09-16T11:44:24.134 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.192:6810/33332", 2024-09-16T11:44:24.134 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:24.134 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:24.134 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:24.134 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "537fa5df-3d0a-4dd4-8155-20ee94e41c82", 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 14, 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 16, 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:24.135 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.136 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.136 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6824", 2024-09-16T11:44:24.136 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:24.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.136 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.136 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6825", 2024-09-16T11:44:24.136 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:24.136 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.136 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6826", 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6827", 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.138 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:24.138 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.138 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.138 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6830", 2024-09-16T11:44:24.138 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:24.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.138 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.138 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6831", 2024-09-16T11:44:24.138 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:24.138 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.138 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6828", 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6829", 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.139 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.140 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.192:6825/33362", 2024-09-16T11:44:24.140 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.192:6827/33362", 2024-09-16T11:44:24.140 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.192:6831/33362", 2024-09-16T11:44:24.140 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.192:6829/33362", 2024-09-16T11:44:24.140 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:24.140 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:24.140 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:24.140 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.140 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.140 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:24.140 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2024-09-16T11:44:24.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T11:44:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.581514+0000", 2024-09-16T11:44:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T11:44:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:24.141 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.674271+0000", 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:24.142 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:10.545402+0000", 2024-09-16T11:44:24.143 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:24.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.143 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T11:44:24.143 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:24.143 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:24.143 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:24.143 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:24.143 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:24.143 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.733565+0000", 2024-09-16T11:44:24.143 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:24.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.144 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T11:44:24.144 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:24.144 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:24.144 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:24.144 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:24.144 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:24.144 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.623479+0000", 2024-09-16T11:44:24.144 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:24.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.144 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T11:44:24.144 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:24.144 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:24.145 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:24.145 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:24.145 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:24.145 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.730066+0000", 2024-09-16T11:44:24.145 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:24.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.145 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T11:44:24.145 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:24.145 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:24.145 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:24.145 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:24.145 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:24.146 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:10.556650+0000", 2024-09-16T11:44:24.146 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:24.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.146 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T11:44:24.146 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:24.146 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:24.146 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:24.146 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:24.146 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:24.146 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.702368+0000", 2024-09-16T11:44:24.146 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:24.146 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.147 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:24.147 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2024-09-16T11:44:24.147 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2024-09-16T11:44:24.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.147 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.8", 2024-09-16T11:44:24.147 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T11:44:24.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.147 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 5, 2024-09-16T11:44:24.147 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 1 2024-09-16T11:44:24.147 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.147 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.147 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.147 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:24.148 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2024-09-16T11:44:24.148 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2024-09-16T11:44:24.148 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2024-09-16T11:44:24.148 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2024-09-16T11:44:24.148 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2024-09-16T11:44:24.148 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2024-09-16T11:44:24.148 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2024-09-16T11:44:24.148 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2024-09-16T11:44:24.148 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2024-09-16T11:44:24.148 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2024-09-16T11:44:24.148 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2024-09-16T11:44:24.148 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.149 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2024-09-16T11:44:24.149 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2024-09-16T11:44:24.149 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2024-09-16T11:44:24.149 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2024-09-16T11:44:24.149 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2024-09-16T11:44:24.149 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2024-09-16T11:44:24.149 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2024-09-16T11:44:24.149 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2024-09-16T11:44:24.149 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2024-09-16T11:44:24.149 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2024-09-16T11:44:24.149 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2024-09-16T11:44:24.150 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.150 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:24.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:24.153 INFO:tasks.thrashosds.thrasher:j is {'epoch': 73, 'fsid': '3420bcad-625f-4e15-aaff-fb4236322168', 'created': '2024-09-16T11:43:01.654626+0000', 'modified': '2024-09-16T11:44:23.963139+0000', 'last_up_change': '2024-09-16T11:43:11.689589+0000', 'last_in_change': '2024-09-16T11:43:04.526236+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T11:43:11.406582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T11:43:13.125709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '23', 'last_force_op_resend_prenautilus': '23', 'last_force_op_resend_preluminous': '23', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-09-16T11:43:21.227888+0000', 'flags': 8201, 'flags_names': 'hashpspool,incomplete_clones,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '23', 'last_force_op_resend_prenautilus': '23', 'last_force_op_resend_preluminous': '23', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T11:43:30.105533+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 37, 'snap_epoch': 73, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': '91f052e4-4c8f-4484-92ce-bd1a7f8f3efb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6800', 'nonce': 33355}, {'type': 'v1', 'addr': '172.21.15.70:6801', 'nonce': 33355}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6802', 'nonce': 33355}, {'type': 'v1', 'addr': '172.21.15.70:6803', 'nonce': 33355}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6806', 'nonce': 33355}, {'type': 'v1', 'addr': '172.21.15.70:6807', 'nonce': 33355}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6804', 'nonce': 33355}, {'type': 'v1', 'addr': '172.21.15.70:6805', 'nonce': 33355}]}, 'public_addr': '172.21.15.70:6801/33355', 'cluster_addr': '172.21.15.70:6803/33355', 'heartbeat_back_addr': '172.21.15.70:6807/33355', 'heartbeat_front_addr': '172.21.15.70:6805/33355', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '568b1280-7365-4094-afe8-b0b5e766de6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6816', 'nonce': 33359}, {'type': 'v1', 'addr': '172.21.15.70:6817', 'nonce': 33359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6818', 'nonce': 33359}, {'type': 'v1', 'addr': '172.21.15.70:6819', 'nonce': 33359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6822', 'nonce': 33359}, {'type': 'v1', 'addr': '172.21.15.70:6823', 'nonce': 33359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6820', 'nonce': 33359}, {'type': 'v1', 'addr': '172.21.15.70:6821', 'nonce': 33359}]}, 'public_addr': '172.21.15.70:6817/33359', 'cluster_addr': '172.21.15.70:6819/33359', 'heartbeat_back_addr': '172.21.15.70:6823/33359', 'heartbeat_front_addr': '172.21.15.70:6821/33359', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '328adfa3-c1b4-4745-9759-7327c8cbd499', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6808', 'nonce': 33356}, {'type': 'v1', 'addr': '172.21.15.70:6809', 'nonce': 33356}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6810', 'nonce': 33356}, {'type': 'v1', 'addr': '172.21.15.70:6811', 'nonce': 33356}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6814', 'nonce': 33356}, {'type': 'v1', 'addr': '172.21.15.70:6815', 'nonce': 33356}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6812', 'nonce': 33356}, {'type': 'v1', 'addr': '172.21.15.70:6813', 'nonce': 33356}]}, 'public_addr': '172.21.15.70:6809/33356', 'cluster_addr': '172.21.15.70:6811/33356', 'heartbeat_back_addr': '172.21.15.70:6815/33356', 'heartbeat_front_addr': '172.21.15.70:6813/33356', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '6df66ae7-e835-4d8f-bd27-243d7a9705b5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6824', 'nonce': 33384}, {'type': 'v1', 'addr': '172.21.15.70:6825', 'nonce': 33384}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6826', 'nonce': 33384}, {'type': 'v1', 'addr': '172.21.15.70:6827', 'nonce': 33384}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6830', 'nonce': 33384}, {'type': 'v1', 'addr': '172.21.15.70:6831', 'nonce': 33384}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6828', 'nonce': 33384}, {'type': 'v1', 'addr': '172.21.15.70:6829', 'nonce': 33384}]}, 'public_addr': '172.21.15.70:6825/33384', 'cluster_addr': '172.21.15.70:6827/33384', 'heartbeat_back_addr': '172.21.15.70:6831/33384', 'heartbeat_front_addr': '172.21.15.70:6829/33384', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '29f52d62-d4c3-49f9-992f-23817f5622d2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 33331}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 33331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6805', 'nonce': 33331}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 33331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6813', 'nonce': 33331}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 33331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6809', 'nonce': 33331}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 33331}]}, 'public_addr': '172.21.15.192:6803/33331', 'cluster_addr': '172.21.15.192:6807/33331', 'heartbeat_back_addr': '172.21.15.192:6815/33331', 'heartbeat_front_addr': '172.21.15.192:6811/33331', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0c1a275a-3c75-4e0b-bd35-d34b64d99c0a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 33333}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 33333}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 33333}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 33333}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 33333}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 33333}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 33333}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 33333}]}, 'public_addr': '172.21.15.192:6817/33333', 'cluster_addr': '172.21.15.192:6819/33333', 'heartbeat_back_addr': '172.21.15.192:6823/33333', 'heartbeat_front_addr': '172.21.15.192:6821/33333', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd79f0070-ce58-48e7-a56a-2762ec76a271', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6801', 'nonce': 33332}, {'type': 'v1', 'addr': '172.21.15.192:6802', 'nonce': 33332}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 33332}, {'type': 'v1', 'addr': '172.21.15.192:6806', 'nonce': 33332}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 33332}, {'type': 'v1', 'addr': '172.21.15.192:6814', 'nonce': 33332}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 33332}, {'type': 'v1', 'addr': '172.21.15.192:6810', 'nonce': 33332}]}, 'public_addr': '172.21.15.192:6802/33332', 'cluster_addr': '172.21.15.192:6806/33332', 'heartbeat_back_addr': '172.21.15.192:6814/33332', 'heartbeat_front_addr': '172.21.15.192:6810/33332', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '537fa5df-3d0a-4dd4-8155-20ee94e41c82', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 16, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6824', 'nonce': 33362}, {'type': 'v1', 'addr': '172.21.15.192:6825', 'nonce': 33362}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6826', 'nonce': 33362}, {'type': 'v1', 'addr': '172.21.15.192:6827', 'nonce': 33362}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6830', 'nonce': 33362}, {'type': 'v1', 'addr': '172.21.15.192:6831', 'nonce': 33362}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6828', 'nonce': 33362}, {'type': 'v1', 'addr': '172.21.15.192:6829', 'nonce': 33362}]}, 'public_addr': '172.21.15.192:6825/33362', 'cluster_addr': '172.21.15.192:6827/33362', 'heartbeat_back_addr': '172.21.15.192:6831/33362', 'heartbeat_front_addr': '172.21.15.192:6829/33362', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.581514+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.674271+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:10.545402+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.733565+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.623479+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.730066+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:10.556650+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.702368+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '4.8', 'mappings': [{'from': 5, 'to': 1}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T11:44:24.153 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:44:24.154 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-16T11:44:24.155 INFO:teuthology.orchestra.run.smithi070.stdout:5263 read 0x345639c thru 0x3464ca7 (0xe90c bytes) 2024-09-16T11:44:24.155 INFO:teuthology.orchestra.run.smithi070.stdout:5264 writesame 0xb6e9632 thru 0xb6f2aa9 data_size 0x4(0x9478 bytes) 2024-09-16T11:44:24.155 INFO:teuthology.orchestra.run.smithi070.stdout:5265 writesame 0xc10bbeb thru 0xc10ec46 data_size 0xa(0x305c bytes) 2024-09-16T11:44:24.155 INFO:teuthology.orchestra.run.smithi070.stdout:5266 punch from 0x7eaaa52 to 0x7eb79f4, (0xcfa2 bytes) 2024-09-16T11:44:24.155 INFO:teuthology.orchestra.run.smithi070.stdout:5267 punch from 0xb1de4b4 to 0xb1ed12e, (0xec7a bytes) 2024-09-16T11:44:24.155 INFO:teuthology.orchestra.run.smithi070.stdout:5268 write 0x77f7808 thru 0x77ff3e8 (0x7be1 bytes) 2024-09-16T11:44:24.155 INFO:teuthology.orchestra.run.smithi070.stdout:5269 writesame 0x8b0ae01 thru 0x8b14c89 data_size 0x1(0x9e89 bytes) 2024-09-16T11:44:24.155 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:24.155 INFO:teuthology.orchestra.run.smithi070.stdout:5271 punch from 0x9c8af9b to 0x9c90438, (0x549d bytes) 2024-09-16T11:44:24.155 INFO:teuthology.orchestra.run.smithi070.stdout:5272 read 0x4e736f0 thru 0x4e801fb (0xcb0c bytes) 2024-09-16T11:44:24.155 INFO:teuthology.orchestra.run.smithi070.stdout:5275 writesame 0x4df89b6 thru 0x4e02a7a data_size 0x11(0xa0c5 bytes) 2024-09-16T11:44:24.155 INFO:teuthology.orchestra.run.smithi070.stdout:5277 punch from 0x9782c3f to 0x97926ce, (0xfa8f bytes) 2024-09-16T11:44:24.156 INFO:teuthology.orchestra.run.smithi070.stdout:5278 writesame 0x13a5675 thru 0x13a74ec data_size 0x34(0x1e78 bytes) 2024-09-16T11:44:24.189 INFO:teuthology.orchestra.run.smithi070.stdout:5280 trunc from 0xc10ec47 to 0xa4d3c2 2024-09-16T11:44:24.191 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:24.189+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:24.253 INFO:teuthology.orchestra.run.smithi070.stdout:5282 read 0x504c thru 0x12aaa (0xda5f bytes) 2024-09-16T11:44:24.268 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:24.266+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:24.276 INFO:teuthology.orchestra.run.smithi070.stdout:5285 writesame 0xd1b3f56 thru 0xd1b6ed9 data_size 0x2(0x2f84 bytes) 2024-09-16T11:44:24.285 INFO:teuthology.orchestra.run.smithi070.stdout:5287 trunc from 0xd1b6eda to 0x65f8124 2024-09-16T11:44:24.306 INFO:teuthology.orchestra.run.smithi070.stdout:5288 writesame 0xcb945bb thru 0xcb9b08b data_size 0x5(0x6ad1 bytes) 2024-09-16T11:44:24.306 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:24.313 INFO:teuthology.orchestra.run.smithi070.stdout:5293 write 0xd5993db thru 0xd59b8e5 (0x250b bytes) 2024-09-16T11:44:24.316 INFO:teuthology.orchestra.run.smithi070.stdout:5296 punch from 0xca830 to 0xcc412, (0x1be2 bytes) 2024-09-16T11:44:24.317 INFO:teuthology.orchestra.run.smithi070.stdout:5297 read 0x2d0ae30 thru 0x2d154f2 (0xa6c3 bytes) 2024-09-16T11:44:24.317 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:24.317 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:24.318 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:24.318 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:24.318 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:24.318 INFO:teuthology.orchestra.run.smithi070.stdout:5300 read 0x129fa08 thru 0x12a7343 (0x793c bytes) 2024-09-16T11:44:24.327 INFO:teuthology.orchestra.run.smithi070.stdout:5301 trunc from 0xd59b8e6 to 0x5de0e9c 2024-09-16T11:44:24.335 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:24.339 INFO:teuthology.orchestra.run.smithi070.stdout:5302 read 0x949f48 thru 0x94fa51 (0x5b0a bytes) 2024-09-16T11:44:24.339 INFO:teuthology.orchestra.run.smithi070.stdout:5303 write 0x5c86f2c thru 0x5c945a8 (0xd67d bytes) 2024-09-16T11:44:24.343 INFO:teuthology.orchestra.run.smithi070.stdout:5304 punch from 0x32828d4 to 0x3283fa5, (0x16d1 bytes) 2024-09-16T11:44:24.361 INFO:teuthology.orchestra.run.smithi070.stdout:5305 writesame 0xe557de4 thru 0xe55cc2c data_size 0x7(0x4e49 bytes) 2024-09-16T11:44:24.361 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:24.361 INFO:teuthology.orchestra.run.smithi070.stdout:5308 writesame 0x496f8ba thru 0x497df30 data_size 0x1(0xe677 bytes) 2024-09-16T11:44:24.369 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:24.367+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:24.386 INFO:teuthology.orchestra.run.smithi070.stdout:5309 read 0xd2b8317 thru 0xd2c53a6 (0xd090 bytes) 2024-09-16T11:44:24.386 INFO:teuthology.orchestra.run.smithi070.stdout:5311 read 0x60e691e thru 0x60f096f (0xa052 bytes) 2024-09-16T11:44:24.386 INFO:teuthology.orchestra.run.smithi070.stdout:5312 read 0x86a98f8 thru 0x86afbbd (0x62c6 bytes) 2024-09-16T11:44:24.387 INFO:teuthology.orchestra.run.smithi070.stdout:5313 write 0xc3c8df0 thru 0xc3cf988 (0x6b99 bytes) 2024-09-16T11:44:24.390 INFO:teuthology.orchestra.run.smithi070.stdout:5314 clone 35 order 22 su 2097152 sc 6 2024-09-16T11:44:24.443 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:24.444 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:24.444 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:24.444 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:24.444 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:24.452 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:24.493 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:24.491+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:24.505 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:24.506 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:44:24.516 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-16T11:44:23.967702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":904994068,"num_objects":1514,"num_object_clones":222,"num_object_copies":3028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1512,"num_whiteouts":892,"num_read":81689,"num_read_kb":348655,"num_write":14108,"num_write_kb":213170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":8960357,"num_keys_recovered":12,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12114,"ondisk_log_size":12114,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2485520,"kb_used_data":31632,"kb_used_omap":13,"kb_used_meta":2453618,"kb_avail":752489200,"statfs":{"total":773094113280,"available":770548940800,"internally_reserved":0,"allocated":32391168,"data_stored":227464938,"data_compressed":8964601,"data_compressed_allocated":20373504,"data_compressed_original":221356900,"omap_allocated":13632,"internal_metadata":2512505536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":59220124,"num_objects":89,"num_object_clones":20,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":28,"num_read":6538,"num_read_kb":23748,"num_write":1252,"num_write_kb":19189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.584541"},"pg_stats":[{"pgid":"4.1f","version":"68'232","reported_seq":1949,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T11:44:18.957845+0000","last_change":"2024-09-16T11:43:33.782279+0000","last_active":"2024-09-16T11:44:18.957845+0000","last_peered":"2024-09-16T11:44:18.957845+0000","last_clean":"2024-09-16T11:44:18.957845+0000","last_became_active":"2024-09-16T11:43:31.779428+0000","last_became_peered":"2024-09-16T11:43:31.779428+0000","last_unstale":"2024-09-16T11:44:18.957845+0000","last_undegraded":"2024-09-16T11:44:18.957845+0000","last_fullsized":"2024-09-16T11:44:18.957845+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:30.437698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090324899999999996,"stat_sum":{"num_bytes":12308087,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":2524,"num_read_kb":4888,"num_write":228,"num_write_kb":3790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"70'295","reported_seq":1215,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.479184+0000","last_change":"2024-09-16T11:43:34.248024+0000","last_active":"2024-09-16T11:44:20.479184+0000","last_peered":"2024-09-16T11:44:20.479184+0000","last_clean":"2024-09-16T11:44:20.479184+0000","last_became_active":"2024-09-16T11:43:31.889706+0000","last_became_peered":"2024-09-16T11:43:31.889706+0000","last_unstale":"2024-09-16T11:44:20.479184+0000","last_undegraded":"2024-09-16T11:44:20.479184+0000","last_fullsized":"2024-09-16T11:44:20.479184+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:25.242116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027516299999999998,"stat_sum":{"num_bytes":16803312,"num_objects":49,"num_object_clones":10,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":27,"num_read":1458,"num_read_kb":6488,"num_write":347,"num_write_kb":4117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"71'169","reported_seq":1224,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T11:44:22.224083+0000","last_change":"2024-09-16T11:43:33.777888+0000","last_active":"2024-09-16T11:44:22.224083+0000","last_peered":"2024-09-16T11:44:22.224083+0000","last_clean":"2024-09-16T11:44:22.224083+0000","last_became_active":"2024-09-16T11:43:31.775021+0000","last_became_peered":"2024-09-16T11:43:31.775021+0000","last_unstale":"2024-09-16T11:44:22.224083+0000","last_undegraded":"2024-09-16T11:44:22.224083+0000","last_fullsized":"2024-09-16T11:44:22.224083+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:50.045796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075167799999999998,"stat_sum":{"num_bytes":22664607,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":24,"num_read":1052,"num_read_kb":9216,"num_write":175,"num_write_kb":7332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"70'223","reported_seq":1806,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.231867+0000","last_change":"2024-09-16T11:43:33.772949+0000","last_active":"2024-09-16T11:44:20.231867+0000","last_peered":"2024-09-16T11:44:20.231867+0000","last_clean":"2024-09-16T11:44:20.231867+0000","last_became_active":"2024-09-16T11:43:31.773661+0000","last_became_peered":"2024-09-16T11:43:31.773661+0000","last_unstale":"2024-09-16T11:44:20.231867+0000","last_undegraded":"2024-09-16T11:44:20.231867+0000","last_fullsized":"2024-09-16T11:44:20.231867+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:36.062796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023334699999999999,"stat_sum":{"num_bytes":28129958,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":20,"num_read":1798,"num_read_kb":13003,"num_write":231,"num_write_kb":5554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"70'301","reported_seq":2078,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.279624+0000","last_change":"2024-09-16T11:43:33.773384+0000","last_active":"2024-09-16T11:44:20.279624+0000","last_peered":"2024-09-16T11:44:20.279624+0000","last_clean":"2024-09-16T11:44:20.279624+0000","last_became_active":"2024-09-16T11:43:31.774523+0000","last_became_peered":"2024-09-16T11:43:31.774523+0000","last_unstale":"2024-09-16T11:44:20.279624+0000","last_undegraded":"2024-09-16T11:44:20.279624+0000","last_fullsized":"2024-09-16T11:44:20.279624+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:15.703792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038485,"stat_sum":{"num_bytes":27812524,"num_objects":48,"num_object_clones":5,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":31,"num_read":2769,"num_read_kb":13878,"num_write":264,"num_write_kb":3943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"70'450","reported_seq":2672,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.671539+0000","last_change":"2024-09-16T11:43:33.774141+0000","last_active":"2024-09-16T11:44:20.671539+0000","last_peered":"2024-09-16T11:44:20.671539+0000","last_clean":"2024-09-16T11:44:20.671539+0000","last_became_active":"2024-09-16T11:43:31.774794+0000","last_became_peered":"2024-09-16T11:43:31.774794+0000","last_unstale":"2024-09-16T11:44:20.671539+0000","last_undegraded":"2024-09-16T11:44:20.671539+0000","last_fullsized":"2024-09-16T11:44:20.671539+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:06:57.981702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010218040000000001,"stat_sum":{"num_bytes":23268620,"num_objects":63,"num_object_clones":10,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":37,"num_read":3429,"num_read_kb":9416,"num_write":580,"num_write_kb":7504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"71'576","reported_seq":1138,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T11:44:22.223905+0000","last_change":"2024-09-16T11:43:33.777212+0000","last_active":"2024-09-16T11:44:22.223905+0000","last_peered":"2024-09-16T11:44:22.223905+0000","last_clean":"2024-09-16T11:44:22.223905+0000","last_became_active":"2024-09-16T11:43:31.769719+0000","last_became_peered":"2024-09-16T11:43:31.769719+0000","last_unstale":"2024-09-16T11:44:22.223905+0000","last_undegraded":"2024-09-16T11:44:22.223905+0000","last_fullsized":"2024-09-16T11:44:22.223905+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:06.911046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035706099999999999,"stat_sum":{"num_bytes":27930786,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":2369,"num_read_kb":3759,"num_write":795,"num_write_kb":2258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"63'135","reported_seq":967,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T11:44:18.966126+0000","last_change":"2024-09-16T11:43:33.774063+0000","last_active":"2024-09-16T11:44:18.966126+0000","last_peered":"2024-09-16T11:44:18.966126+0000","last_clean":"2024-09-16T11:44:18.966126+0000","last_became_active":"2024-09-16T11:43:31.776951+0000","last_became_peered":"2024-09-16T11:43:31.776951+0000","last_unstale":"2024-09-16T11:44:18.966126+0000","last_undegraded":"2024-09-16T11:44:18.966126+0000","last_fullsized":"2024-09-16T11:44:18.966126+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:28:39.026621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094366700000000003,"stat_sum":{"num_bytes":27102422,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":22,"num_read":828,"num_read_kb":6410,"num_write":118,"num_write_kb":5320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"70'392","reported_seq":1611,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:21.286950+0000","last_change":"2024-09-16T11:43:33.777913+0000","last_active":"2024-09-16T11:44:21.286950+0000","last_peered":"2024-09-16T11:44:21.286950+0000","last_clean":"2024-09-16T11:44:21.286950+0000","last_became_active":"2024-09-16T11:43:31.889807+0000","last_became_peered":"2024-09-16T11:43:31.889807+0000","last_unstale":"2024-09-16T11:44:21.286950+0000","last_undegraded":"2024-09-16T11:44:21.286950+0000","last_fullsized":"2024-09-16T11:44:21.286950+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:30.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071799400000000001,"stat_sum":{"num_bytes":29826926,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":28,"num_read":3308,"num_read_kb":8608,"num_write":445,"num_write_kb":5674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"70'340","reported_seq":1595,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.665678+0000","last_change":"2024-09-16T11:43:34.248043+0000","last_active":"2024-09-16T11:44:20.665678+0000","last_peered":"2024-09-16T11:44:20.665678+0000","last_clean":"2024-09-16T11:44:20.665678+0000","last_became_active":"2024-09-16T11:43:31.776111+0000","last_became_peered":"2024-09-16T11:43:31.776111+0000","last_unstale":"2024-09-16T11:44:20.665678+0000","last_undegraded":"2024-09-16T11:44:20.665678+0000","last_fullsized":"2024-09-16T11:44:20.665678+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:21:28.658052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024883199999999999,"stat_sum":{"num_bytes":27266325,"num_objects":44,"num_object_clones":5,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":29,"num_read":2033,"num_read_kb":16273,"num_write":330,"num_write_kb":11762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"71'539","reported_seq":2238,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T11:44:22.223434+0000","last_change":"2024-09-16T11:43:33.785207+0000","last_active":"2024-09-16T11:44:22.223434+0000","last_peered":"2024-09-16T11:44:22.223434+0000","last_clean":"2024-09-16T11:44:22.223434+0000","last_became_active":"2024-09-16T11:43:31.889629+0000","last_became_peered":"2024-09-16T11:43:31.889629+0000","last_unstale":"2024-09-16T11:44:22.223434+0000","last_undegraded":"2024-09-16T11:44:22.223434+0000","last_fullsized":"2024-09-16T11:44:22.223434+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:53.124562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022324199999999999,"stat_sum":{"num_bytes":38189247,"num_objects":66,"num_object_clones":10,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":35,"num_read":3514,"num_read_kb":26123,"num_write":703,"num_write_kb":14958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"70'567","reported_seq":1407,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:21.287816+0000","last_change":"2024-09-16T11:43:33.778129+0000","last_active":"2024-09-16T11:44:21.287816+0000","last_peered":"2024-09-16T11:44:21.287816+0000","last_clean":"2024-09-16T11:44:21.287816+0000","last_became_active":"2024-09-16T11:43:31.773947+0000","last_became_peered":"2024-09-16T11:43:31.773947+0000","last_unstale":"2024-09-16T11:44:21.287816+0000","last_undegraded":"2024-09-16T11:44:21.287816+0000","last_fullsized":"2024-09-16T11:44:21.287816+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:36:48.418141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085420199999999998,"stat_sum":{"num_bytes":33959901,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":27,"num_read":2788,"num_read_kb":12802,"num_write":840,"num_write_kb":8645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"70'416","reported_seq":1934,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.525950+0000","last_change":"2024-09-16T11:43:33.773601+0000","last_active":"2024-09-16T11:44:20.525950+0000","last_peered":"2024-09-16T11:44:20.525950+0000","last_clean":"2024-09-16T11:44:20.525950+0000","last_became_active":"2024-09-16T11:43:31.776781+0000","last_became_peered":"2024-09-16T11:43:31.776781+0000","last_unstale":"2024-09-16T11:44:20.525950+0000","last_undegraded":"2024-09-16T11:44:20.525950+0000","last_fullsized":"2024-09-16T11:44:20.525950+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:14.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056674299999999998,"stat_sum":{"num_bytes":36965426,"num_objects":56,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":32,"num_read":3352,"num_read_kb":14678,"num_write":502,"num_write_kb":10335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"70'240","reported_seq":1563,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.961912+0000","last_change":"2024-09-16T11:43:33.773531+0000","last_active":"2024-09-16T11:44:20.961912+0000","last_peered":"2024-09-16T11:44:20.961912+0000","last_clean":"2024-09-16T11:44:20.961912+0000","last_became_active":"2024-09-16T11:43:31.780164+0000","last_became_peered":"2024-09-16T11:43:31.780164+0000","last_unstale":"2024-09-16T11:44:20.961912+0000","last_undegraded":"2024-09-16T11:44:20.961912+0000","last_fullsized":"2024-09-16T11:44:20.961912+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:56.846891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033807800000000001,"stat_sum":{"num_bytes":18369880,"num_objects":28,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":18,"num_read":1594,"num_read_kb":8183,"num_write":254,"num_write_kb":4526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"71'671","reported_seq":1692,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T11:44:22.331465+0000","last_change":"2024-09-16T11:43:33.777531+0000","last_active":"2024-09-16T11:44:22.331465+0000","last_peered":"2024-09-16T11:44:22.331465+0000","last_clean":"2024-09-16T11:44:22.331465+0000","last_became_active":"2024-09-16T11:43:31.889754+0000","last_became_peered":"2024-09-16T11:43:31.889754+0000","last_unstale":"2024-09-16T11:44:22.331465+0000","last_undegraded":"2024-09-16T11:44:22.331465+0000","last_fullsized":"2024-09-16T11:44:22.331465+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:54.752770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062624299999999996,"stat_sum":{"num_bytes":22396270,"num_objects":59,"num_object_clones":5,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":42,"num_read":3731,"num_read_kb":16163,"num_write":953,"num_write_kb":7833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'286","reported_seq":1418,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T11:44:22.157127+0000","last_change":"2024-09-16T11:43:33.777407+0000","last_active":"2024-09-16T11:44:22.157127+0000","last_peered":"2024-09-16T11:44:22.157127+0000","last_clean":"2024-09-16T11:44:22.157127+0000","last_became_active":"2024-09-16T11:43:31.775962+0000","last_became_peered":"2024-09-16T11:43:31.775962+0000","last_unstale":"2024-09-16T11:44:22.157127+0000","last_undegraded":"2024-09-16T11:44:22.157127+0000","last_fullsized":"2024-09-16T11:44:22.157127+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:46.492489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062432399999999999,"stat_sum":{"num_bytes":15641698,"num_objects":44,"num_object_clones":6,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":31,"num_read":2550,"num_read_kb":8170,"num_write":267,"num_write_kb":1655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T11:44:18.957818+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:44:18.957818+0000","last_peered":"2024-09-16T11:44:18.957818+0000","last_clean":"2024-09-16T11:44:18.957818+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:44:18.957818+0000","last_undegraded":"2024-09-16T11:44:18.957818+0000","last_fullsized":"2024-09-16T11:44:18.957818+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"70'276","reported_seq":1322,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.507225+0000","last_change":"2024-09-16T11:43:33.773359+0000","last_active":"2024-09-16T11:44:20.507225+0000","last_peered":"2024-09-16T11:44:20.507225+0000","last_clean":"2024-09-16T11:44:20.507225+0000","last_became_active":"2024-09-16T11:43:31.777591+0000","last_became_peered":"2024-09-16T11:43:31.777591+0000","last_unstale":"2024-09-16T11:44:20.507225+0000","last_undegraded":"2024-09-16T11:44:20.507225+0000","last_fullsized":"2024-09-16T11:44:20.507225+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:00:39.586913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037893300000000003,"stat_sum":{"num_bytes":22585392,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":23,"num_read":2191,"num_read_kb":8156,"num_write":241,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":65,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:19.957726+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:44:19.957726+0000","last_peered":"2024-09-16T11:44:19.957726+0000","last_clean":"2024-09-16T11:44:19.957726+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:44:19.957726+0000","last_undegraded":"2024-09-16T11:44:19.957726+0000","last_fullsized":"2024-09-16T11:44:19.957726+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"71'247","reported_seq":2463,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T11:44:22.223733+0000","last_change":"2024-09-16T11:43:33.777644+0000","last_active":"2024-09-16T11:44:22.223733+0000","last_peered":"2024-09-16T11:44:22.223733+0000","last_clean":"2024-09-16T11:44:22.223733+0000","last_became_active":"2024-09-16T11:43:31.892159+0000","last_became_peered":"2024-09-16T11:43:31.892159+0000","last_unstale":"2024-09-16T11:44:22.223733+0000","last_undegraded":"2024-09-16T11:44:22.223733+0000","last_fullsized":"2024-09-16T11:44:22.223733+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:42:14.042476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062271400000000001,"stat_sum":{"num_bytes":34903544,"num_objects":55,"num_object_clones":10,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":30,"num_read":2939,"num_read_kb":13885,"num_write":217,"num_write_kb":5905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":65,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:19.956228+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:44:19.956228+0000","last_peered":"2024-09-16T11:44:19.956228+0000","last_clean":"2024-09-16T11:44:19.956228+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:44:19.956228+0000","last_undegraded":"2024-09-16T11:44:19.956228+0000","last_fullsized":"2024-09-16T11:44:19.956228+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"70'289","reported_seq":597,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.525074+0000","last_change":"2024-09-16T11:43:34.247771+0000","last_active":"2024-09-16T11:44:20.525074+0000","last_peered":"2024-09-16T11:44:20.525074+0000","last_clean":"2024-09-16T11:44:20.525074+0000","last_became_active":"2024-09-16T11:43:31.777896+0000","last_became_peered":"2024-09-16T11:43:31.777896+0000","last_unstale":"2024-09-16T11:44:20.525074+0000","last_undegraded":"2024-09-16T11:44:20.525074+0000","last_fullsized":"2024-09-16T11:44:20.525074+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:35.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026844500000000002,"stat_sum":{"num_bytes":42542030,"num_objects":38,"num_object_clones":5,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":23,"num_read":351,"num_read_kb":5424,"num_write":287,"num_write_kb":5102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":65,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:19.957703+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:44:19.957703+0000","last_peered":"2024-09-16T11:44:19.957703+0000","last_clean":"2024-09-16T11:44:19.957703+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:44:19.957703+0000","last_undegraded":"2024-09-16T11:44:19.957703+0000","last_fullsized":"2024-09-16T11:44:19.957703+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"71'320","reported_seq":735,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T11:44:22.409100+0000","last_change":"2024-09-16T11:43:33.774367+0000","last_active":"2024-09-16T11:44:22.409100+0000","last_peered":"2024-09-16T11:44:22.409100+0000","last_clean":"2024-09-16T11:44:22.409100+0000","last_became_active":"2024-09-16T11:43:31.778506+0000","last_became_peered":"2024-09-16T11:43:31.778506+0000","last_unstale":"2024-09-16T11:44:22.409100+0000","last_undegraded":"2024-09-16T11:44:22.409100+0000","last_fullsized":"2024-09-16T11:44:22.409100+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:12.232493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010404660000000001,"stat_sum":{"num_bytes":5690215,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":1354,"num_read_kb":3456,"num_write":453,"num_write_kb":1994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":57,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T11:44:18.966313+0000","last_change":"2024-09-16T11:43:22.438212+0000","last_active":"2024-09-16T11:44:18.966313+0000","last_peered":"2024-09-16T11:44:18.966313+0000","last_clean":"2024-09-16T11:44:18.966313+0000","last_became_active":"2024-09-16T11:43:22.437868+0000","last_became_peered":"2024-09-16T11:43:22.437868+0000","last_unstale":"2024-09-16T11:44:18.966313+0000","last_undegraded":"2024-09-16T11:44:18.966313+0000","last_fullsized":"2024-09-16T11:44:18.966313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:35.498933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":67,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:19.954717+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:44:19.954717+0000","last_peered":"2024-09-16T11:44:19.954717+0000","last_clean":"2024-09-16T11:44:19.954717+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:44:19.954717+0000","last_undegraded":"2024-09-16T11:44:19.954717+0000","last_fullsized":"2024-09-16T11:44:19.954717+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"71'641","reported_seq":2162,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T11:44:22.230655+0000","last_change":"2024-09-16T11:43:33.774349+0000","last_active":"2024-09-16T11:44:22.230655+0000","last_peered":"2024-09-16T11:44:22.230655+0000","last_clean":"2024-09-16T11:44:22.230655+0000","last_became_active":"2024-09-16T11:43:31.777796+0000","last_became_peered":"2024-09-16T11:43:31.777796+0000","last_unstale":"2024-09-16T11:44:22.230655+0000","last_undegraded":"2024-09-16T11:44:22.230655+0000","last_fullsized":"2024-09-16T11:44:22.230655+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:33.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033368800000000001,"stat_sum":{"num_bytes":41010956,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":25,"num_read":3916,"num_read_kb":14405,"num_write":846,"num_write_kb":9754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":56,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:19.957597+0000","last_change":"2024-09-16T11:43:22.440136+0000","last_active":"2024-09-16T11:44:19.957597+0000","last_peered":"2024-09-16T11:44:19.957597+0000","last_clean":"2024-09-16T11:44:19.957597+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:44:19.957597+0000","last_undegraded":"2024-09-16T11:44:19.957597+0000","last_fullsized":"2024-09-16T11:44:19.957597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:18.461948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T11:44:18.950269+0000","last_change":"2024-09-16T11:43:16.732834+0000","last_active":"2024-09-16T11:44:18.950269+0000","last_peered":"2024-09-16T11:44:18.950269+0000","last_clean":"2024-09-16T11:44:18.950269+0000","last_became_active":"2024-09-16T11:43:14.724728+0000","last_became_peered":"2024-09-16T11:43:14.724728+0000","last_unstale":"2024-09-16T11:44:18.950269+0000","last_undegraded":"2024-09-16T11:44:18.950269+0000","last_fullsized":"2024-09-16T11:44:18.950269+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:47.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"71'450","reported_seq":2877,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T11:44:22.153784+0000","last_change":"2024-09-16T11:43:33.774126+0000","last_active":"2024-09-16T11:44:22.153784+0000","last_peered":"2024-09-16T11:44:22.153784+0000","last_clean":"2024-09-16T11:44:22.153784+0000","last_became_active":"2024-09-16T11:43:31.777967+0000","last_became_peered":"2024-09-16T11:43:31.777967+0000","last_unstale":"2024-09-16T11:44:22.153784+0000","last_undegraded":"2024-09-16T11:44:22.153784+0000","last_fullsized":"2024-09-16T11:44:22.153784+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:43.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000373488,"stat_sum":{"num_bytes":54804694,"num_objects":60,"num_object_clones":12,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":31,"num_read":3372,"num_read_kb":18364,"num_write":555,"num_write_kb":6794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"56'1","reported_seq":57,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T11:44:18.959511+0000","last_change":"2024-09-16T11:43:22.437965+0000","last_active":"2024-09-16T11:44:18.959511+0000","last_peered":"2024-09-16T11:44:18.959511+0000","last_clean":"2024-09-16T11:44:18.959511+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:44:18.959511+0000","last_undegraded":"2024-09-16T11:44:18.959511+0000","last_fullsized":"2024-09-16T11:44:18.959511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:52.884360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":65,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:19.956270+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:44:19.956270+0000","last_peered":"2024-09-16T11:44:19.956270+0000","last_clean":"2024-09-16T11:44:19.956270+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:44:19.956270+0000","last_undegraded":"2024-09-16T11:44:19.956270+0000","last_fullsized":"2024-09-16T11:44:19.956270+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"70'214","reported_seq":898,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.437408+0000","last_change":"2024-09-16T11:43:34.248254+0000","last_active":"2024-09-16T11:44:20.437408+0000","last_peered":"2024-09-16T11:44:20.437408+0000","last_clean":"2024-09-16T11:44:20.437408+0000","last_became_active":"2024-09-16T11:43:31.776645+0000","last_became_peered":"2024-09-16T11:43:31.776645+0000","last_unstale":"2024-09-16T11:44:20.437408+0000","last_undegraded":"2024-09-16T11:44:20.437408+0000","last_fullsized":"2024-09-16T11:44:20.437408+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:53.198323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038670399999999998,"stat_sum":{"num_bytes":45980109,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":29,"num_read":715,"num_read_kb":4325,"num_write":186,"num_write_kb":3219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":57,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:19.954130+0000","last_change":"2024-09-16T11:43:22.587076+0000","last_active":"2024-09-16T11:44:19.954130+0000","last_peered":"2024-09-16T11:44:19.954130+0000","last_clean":"2024-09-16T11:44:19.954130+0000","last_became_active":"2024-09-16T11:43:22.586861+0000","last_became_peered":"2024-09-16T11:43:22.586861+0000","last_unstale":"2024-09-16T11:44:19.954130+0000","last_undegraded":"2024-09-16T11:44:19.954130+0000","last_fullsized":"2024-09-16T11:44:19.954130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:07:45.296752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":66,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:19.954749+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:44:19.954749+0000","last_peered":"2024-09-16T11:44:19.954749+0000","last_clean":"2024-09-16T11:44:19.954749+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:44:19.954749+0000","last_undegraded":"2024-09-16T11:44:19.954749+0000","last_fullsized":"2024-09-16T11:44:19.954749+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":152,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:19.956278+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:44:19.956278+0000","last_peered":"2024-09-16T11:44:19.956278+0000","last_clean":"2024-09-16T11:44:19.956278+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:44:19.956278+0000","last_undegraded":"2024-09-16T11:44:19.956278+0000","last_fullsized":"2024-09-16T11:44:19.956278+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"70'417","reported_seq":2986,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.338063+0000","last_change":"2024-09-16T11:43:33.781611+0000","last_active":"2024-09-16T11:44:20.338063+0000","last_peered":"2024-09-16T11:44:20.338063+0000","last_clean":"2024-09-16T11:44:20.338063+0000","last_became_active":"2024-09-16T11:43:31.782071+0000","last_became_peered":"2024-09-16T11:43:31.782071+0000","last_unstale":"2024-09-16T11:44:20.338063+0000","last_undegraded":"2024-09-16T11:44:20.338063+0000","last_fullsized":"2024-09-16T11:44:20.338063+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:15.516215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033637000000000001,"stat_sum":{"num_bytes":64757063,"num_objects":58,"num_object_clones":9,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":27,"num_read":3548,"num_read_kb":23518,"num_write":466,"num_write_kb":7724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"71'529","reported_seq":3169,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T11:44:22.107313+0000","last_change":"2024-09-16T11:44:10.921411+0000","last_active":"2024-09-16T11:44:22.107313+0000","last_peered":"2024-09-16T11:44:22.107313+0000","last_clean":"2024-09-16T11:44:22.107313+0000","last_became_active":"2024-09-16T11:44:10.920131+0000","last_became_peered":"2024-09-16T11:44:10.920131+0000","last_unstale":"2024-09-16T11:44:22.107313+0000","last_undegraded":"2024-09-16T11:44:22.107313+0000","last_fullsized":"2024-09-16T11:44:22.107313+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:55:10.158414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398253,"stat_sum":{"num_bytes":46062606,"num_objects":71,"num_object_clones":11,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":40,"num_read":3865,"num_read_kb":29720,"num_write":607,"num_write_kb":17101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":8960357,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"70'546","reported_seq":2801,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.223365+0000","last_change":"2024-09-16T11:43:33.773950+0000","last_active":"2024-09-16T11:44:20.223365+0000","last_peered":"2024-09-16T11:44:20.223365+0000","last_clean":"2024-09-16T11:44:20.223365+0000","last_became_active":"2024-09-16T11:43:31.775309+0000","last_became_peered":"2024-09-16T11:43:31.775309+0000","last_unstale":"2024-09-16T11:44:20.223365+0000","last_undegraded":"2024-09-16T11:44:20.223365+0000","last_fullsized":"2024-09-16T11:44:20.223365+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:23.106465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086187000000000004,"stat_sum":{"num_bytes":34608129,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":32,"num_read":6249,"num_read_kb":7576,"num_write":513,"num_write_kb":4994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"70'385","reported_seq":1219,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.601180+0000","last_change":"2024-09-16T11:43:33.781852+0000","last_active":"2024-09-16T11:44:20.601180+0000","last_peered":"2024-09-16T11:44:20.601180+0000","last_clean":"2024-09-16T11:44:20.601180+0000","last_became_active":"2024-09-16T11:43:31.779384+0000","last_became_peered":"2024-09-16T11:43:31.779384+0000","last_unstale":"2024-09-16T11:44:20.601180+0000","last_undegraded":"2024-09-16T11:44:20.601180+0000","last_fullsized":"2024-09-16T11:44:20.601180+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:34.933146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029212900000000002,"stat_sum":{"num_bytes":18034873,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":36,"num_read":1738,"num_read_kb":8424,"num_write":387,"num_write_kb":5617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"70'291","reported_seq":1108,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.263772+0000","last_change":"2024-09-16T11:43:34.247758+0000","last_active":"2024-09-16T11:44:20.263772+0000","last_peered":"2024-09-16T11:44:20.263772+0000","last_clean":"2024-09-16T11:44:20.263772+0000","last_became_active":"2024-09-16T11:43:31.775335+0000","last_became_peered":"2024-09-16T11:43:31.775335+0000","last_unstale":"2024-09-16T11:44:20.263772+0000","last_undegraded":"2024-09-16T11:44:20.263772+0000","last_fullsized":"2024-09-16T11:44:20.263772+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:07.521713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026563699999999999,"stat_sum":{"num_bytes":20232336,"num_objects":41,"num_object_clones":4,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":24,"num_read":1949,"num_read_kb":3200,"num_write":343,"num_write_kb":6990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"70'349","reported_seq":1501,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.338244+0000","last_change":"2024-09-16T11:43:33.773570+0000","last_active":"2024-09-16T11:44:20.338244+0000","last_peered":"2024-09-16T11:44:20.338244+0000","last_clean":"2024-09-16T11:44:20.338244+0000","last_became_active":"2024-09-16T11:43:31.774601+0000","last_became_peered":"2024-09-16T11:43:31.774601+0000","last_unstale":"2024-09-16T11:44:20.338244+0000","last_undegraded":"2024-09-16T11:44:20.338244+0000","last_fullsized":"2024-09-16T11:44:20.338244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:12.584420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055828300000000002,"stat_sum":{"num_bytes":12646915,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":28,"num_read":2624,"num_read_kb":3998,"num_write":328,"num_write_kb":2169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"70'536","reported_seq":1560,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.019482+0000","last_change":"2024-09-16T11:43:33.773852+0000","last_active":"2024-09-16T11:44:20.019482+0000","last_peered":"2024-09-16T11:44:20.019482+0000","last_clean":"2024-09-16T11:44:20.019482+0000","last_became_active":"2024-09-16T11:43:31.772506+0000","last_became_peered":"2024-09-16T11:43:31.772506+0000","last_unstale":"2024-09-16T11:44:20.019482+0000","last_undegraded":"2024-09-16T11:44:20.019482+0000","last_fullsized":"2024-09-16T11:44:20.019482+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:36.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076283899999999997,"stat_sum":{"num_bytes":21979272,"num_objects":41,"num_object_clones":4,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":27,"num_read":3095,"num_read_kb":14352,"num_write":746,"num_write_kb":9802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"70'417","reported_seq":1210,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T11:44:20.548960+0000","last_change":"2024-09-16T11:43:33.773770+0000","last_active":"2024-09-16T11:44:20.548960+0000","last_peered":"2024-09-16T11:44:20.548960+0000","last_clean":"2024-09-16T11:44:20.548960+0000","last_became_active":"2024-09-16T11:43:31.779613+0000","last_became_peered":"2024-09-16T11:43:31.779613+0000","last_unstale":"2024-09-16T11:44:20.548960+0000","last_undegraded":"2024-09-16T11:44:20.548960+0000","last_fullsized":"2024-09-16T11:44:20.548960+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:40.995663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000720899,"stat_sum":{"num_bytes":22710916,"num_objects":53,"num_object_clones":9,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":29,"num_read":1862,"num_read_kb":8307,"num_write":508,"num_write_kb":4456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"71'298","reported_seq":1392,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-16T11:44:22.286618+0000","last_change":"2024-09-16T11:43:33.773914+0000","last_active":"2024-09-16T11:44:22.286618+0000","last_peered":"2024-09-16T11:44:22.286618+0000","last_clean":"2024-09-16T11:44:22.286618+0000","last_became_active":"2024-09-16T11:43:31.776132+0000","last_became_peered":"2024-09-16T11:43:31.776132+0000","last_unstale":"2024-09-16T11:44:22.286618+0000","last_undegraded":"2024-09-16T11:44:22.286618+0000","last_fullsized":"2024-09-16T11:44:22.286618+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:53.489198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023678600000000001,"stat_sum":{"num_bytes":7218642,"num_objects":40,"num_object_clones":4,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":25,"num_read":2758,"num_read_kb":3432,"num_write":290,"num_write_kb":5337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":904403681,"num_objects":1508,"num_object_clones":222,"num_object_copies":3016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1508,"num_whiteouts":890,"num_read":81623,"num_read_kb":348600,"num_write":14005,"num_write_kb":211540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":8960357,"num_keys_recovered":12,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26411008,"data_stored":225775933,"data_compressed":9371513,"data_compressed_allocated":20824064,"data_compressed_original":222768490,"omap_allocated":0,"internal_metadata":0},"log_size":12037,"ondisk_log_size":12037,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314316,"kb_used_data":4484,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94057524,"statfs":{"total":96636764160,"available":96314904576,"internally_reserved":0,"allocated":4591616,"data_stored":37170219,"data_compressed":1196738,"data_compressed_allocated":3125248,"data_compressed_original":36432813,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.318},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.33800000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":3.2469999999999999,"5min":3.2469999999999999,"15min":3.2469999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.41399999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.309}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.5640000000000001,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":0.38800000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.36299999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.441}]},{"osd":6,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.377},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.32500000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313252,"kb_used_data":3416,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058588,"statfs":{"total":96636764160,"available":96315994112,"internally_reserved":0,"allocated":3497984,"data_stored":23314111,"data_compressed":1177994,"data_compressed_allocated":2060288,"data_compressed_original":22565882,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.27400000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.374},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.29099999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.42699999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.30399999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.252},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.317}]},{"osd":6,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.36499999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.437}]}]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310488,"kb_used_data":4812,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94061352,"statfs":{"total":96636764160,"available":96318824448,"internally_reserved":0,"allocated":4927488,"data_stored":35450425,"data_compressed":1463821,"data_compressed_allocated":3252224,"data_compressed_original":34598178,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.33300000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.50600000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.35999999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65600000000000003}]},{"osd":5,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.63}]},{"osd":6,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.58899999999999997}]},{"osd":7,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.60899999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311796,"kb_used_data":6032,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94060044,"statfs":{"total":96636764160,"available":96317485056,"internally_reserved":0,"allocated":6176768,"data_stored":43789936,"data_compressed":2091347,"data_compressed_allocated":4247552,"data_compressed_original":42815331,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.23100000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.315}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.29399999999999998}]},{"osd":4,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36199999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.50800000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.372},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.55700000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.40300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308820,"kb_used_data":3132,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063020,"statfs":{"total":96636764160,"available":96320532480,"internally_reserved":0,"allocated":3207168,"data_stored":24166889,"data_compressed":622785,"data_compressed_allocated":1871872,"data_compressed_original":23508647,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.48899999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.437}]},{"osd":2,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.41899999999999998}]},{"osd":3,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.47399999999999998}]},{"osd":4,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.372},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.40699999999999997}]},{"osd":5,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35199999999999998}]},{"osd":7,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.317},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.38300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302068,"kb_used_data":4712,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069772,"statfs":{"total":96636764160,"available":96327446528,"internally_reserved":0,"allocated":4825088,"data_stored":35959341,"data_compressed":1274919,"data_compressed_allocated":3063808,"data_compressed_original":35033139,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.39500000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.29899999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.26800000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.32400000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57699999999999996}]},{"osd":6,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.48099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312116,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059724,"statfs":{"total":96636764160,"available":96317157376,"internally_reserved":0,"allocated":2330624,"data_stored":12971335,"data_compressed":372838,"data_compressed_allocated":1150976,"data_compressed_original":12382752,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.29099999999999998}]},{"osd":1,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.32700000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.36899999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.42499999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.39900000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.439}]},{"osd":7,"last update":"Mon Sep 16 11:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.46100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312664,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059176,"statfs":{"total":96636764160,"available":96316596224,"internally_reserved":0,"allocated":2834432,"data_stored":14642682,"data_compressed":764159,"data_compressed_allocated":1601536,"data_compressed_original":14020158,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.436}]},{"osd":2,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.34000000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.316},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.54700000000000004}]},{"osd":4,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.45500000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.36799999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.443}]},{"osd":7,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2011136,"data_stored":13930910,"data_compressed":779686,"data_compressed_allocated":1601536,"data_compressed_original":13708862,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5488640,"data_stored":43529265,"data_compressed":2160524,"data_compressed_allocated":4325376,"data_compressed_original":42925923,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1630208,"data_stored":13243281,"data_compressed":469869,"data_compressed_allocated":1249280,"data_compressed_original":13038112,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4308992,"data_stored":35504671,"data_compressed":1573006,"data_compressed_allocated":3387392,"data_compressed_original":35007778,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3997696,"data_stored":35970752,"data_compressed":1284064,"data_compressed_allocated":3088384,"data_compressed_original":35459975,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2703360,"data_stored":22961950,"data_compressed":1203562,"data_compressed_allocated":2088960,"data_compressed_original":22606842,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2465792,"data_stored":24343059,"data_compressed":650443,"data_compressed_allocated":1929216,"data_compressed_original":24067417,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3805184,"data_stored":36292045,"data_compressed":1250359,"data_compressed_allocated":3153920,"data_compressed_original":35953581,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:44:24.520 INFO:tasks.thrashosds.thrasher:Setting 4.14 to [6, 4, 1, 7] 2024-09-16T11:44:24.520 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.14', '6', '4', '1', '7'] 2024-09-16T11:44:24.520 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 4.14 6 4 1 7 2024-09-16T11:44:24.572 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:24.570+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:24.594 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:24.594 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:24.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:24.595 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:24.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.595 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19364 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1171456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.595 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:24.595 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.078495204999999, 2024-09-16T11:44:24.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049001805000000002, 2024-09-16T11:44:24.595 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.595 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.595 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.595 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.595 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.595 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19364 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:24.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:44:24.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513280+0000", 2024-09-16T11:44:24.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7090000000000001e-06 2024-09-16T11:44:24.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513286+0000", 2024-09-16T11:44:24.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5840000000000002e-06 2024-09-16T11:44:24.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513290+0000", 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3680000000000004e-06 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562121+0000", 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048831324000000002 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562195+0000", 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3415000000000001e-05 2024-09-16T11:44:24.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562280+0000", 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4975999999999994e-05 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19365 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1490944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.078475466, 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049335349000000001, 2024-09-16T11:44:24.599 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.600 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.600 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.600 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.600 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.600 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19365 2024-09-16T11:44:24.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.600 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.601 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:24.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.601 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:24.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:24.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:24.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513300+0000", 2024-09-16T11:44:24.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3020000000000002e-06 2024-09-16T11:44:24.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513306+0000", 2024-09-16T11:44:24.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8780000000000003e-06 2024-09-16T11:44:24.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513310+0000", 2024-09-16T11:44:24.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1309999999999999e-06 2024-09-16T11:44:24.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562322+0000", 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049011635999999997 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562544+0000", 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022168000000000001 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562633+0000", 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9350000000000003e-05 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.604 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.605 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.605 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19366 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1916928~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.605 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:24.605 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.078458533999999, 2024-09-16T11:44:24.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049865073000000003, 2024-09-16T11:44:24.605 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.605 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.605 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.605 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.605 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.605 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19366 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.606 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515102+0000", 2024-09-16T11:44:24.607 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017877419999999999 2024-09-16T11:44:24.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.607 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.607 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515106+0000", 2024-09-16T11:44:24.607 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4450000000000001e-06 2024-09-16T11:44:24.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.607 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.607 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515113+0000", 2024-09-16T11:44:24.607 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7680000000000008e-06 2024-09-16T11:44:24.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.608 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.608 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515117+0000", 2024-09-16T11:44:24.608 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:44:24.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.609 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.609 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562756+0000", 2024-09-16T11:44:24.609 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047638579 2024-09-16T11:44:24.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.610 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.610 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563093+0000", 2024-09-16T11:44:24.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000337516 2024-09-16T11:44:24.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.610 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.610 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563180+0000", 2024-09-16T11:44:24.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6420000000000003e-05 2024-09-16T11:44:24.610 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.610 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.610 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19371 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3194880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.076561327, 2024-09-16T11:44:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048948356999999998, 2024-09-16T11:44:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19371 2024-09-16T11:44:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.612 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:24.612 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.613 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.613 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:24.613 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.614 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.614 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:24.614 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:24.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.614 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.614 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.615 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515214+0000", 2024-09-16T11:44:24.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5969999999999999e-06 2024-09-16T11:44:24.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.615 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515219+0000", 2024-09-16T11:44:24.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4559999999999999e-06 2024-09-16T11:44:24.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515223+0000", 2024-09-16T11:44:24.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:44:24.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563991+0000", 2024-09-16T11:44:24.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048768911999999998 2024-09-16T11:44:24.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564063+0000", 2024-09-16T11:44:24.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2010999999999999e-05 2024-09-16T11:44:24.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.617 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564160+0000", 2024-09-16T11:44:24.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6617e-05 2024-09-16T11:44:24.617 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.617 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.617 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.618 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19372 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3620864~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.618 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:24.618 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.07654711, 2024-09-16T11:44:24.618 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049130502, 2024-09-16T11:44:24.618 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.618 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.618 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.618 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.618 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.618 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19372 2024-09-16T11:44:24.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.618 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:24.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:24.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.620 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.620 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:24.620 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:44:24.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.621 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.621 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515228+0000", 2024-09-16T11:44:24.621 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T11:44:24.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.621 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.621 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515233+0000", 2024-09-16T11:44:24.621 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:44:24.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.621 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515237+0000", 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.27e-06 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564199+0000", 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048961787999999999 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564275+0000", 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6001000000000006e-05 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.623 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.623 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564356+0000", 2024-09-16T11:44:24.623 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1941999999999996e-05 2024-09-16T11:44:24.623 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.623 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.623 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.624 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.624 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19373 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3940352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.624 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:24.624 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.076532709999999, 2024-09-16T11:44:24.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049289588000000002, 2024-09-16T11:44:24.625 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.625 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.625 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.625 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.625 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.626 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19373 2024-09-16T11:44:24.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.626 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.626 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.626 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.626 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:24.626 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.627 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.627 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:24.627 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.628 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.628 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515241+0000", 2024-09-16T11:44:24.628 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:44:24.628 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.628 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.628 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.628 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515242+0000", 2024-09-16T11:44:24.628 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.528e-06 2024-09-16T11:44:24.628 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.628 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.628 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.628 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515248+0000", 2024-09-16T11:44:24.629 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:24.629 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.629 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.629 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.629 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515251+0000", 2024-09-16T11:44:24.629 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2480000000000001e-06 2024-09-16T11:44:24.629 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.629 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.629 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.629 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564388+0000", 2024-09-16T11:44:24.629 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049137313000000002 2024-09-16T11:44:24.629 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564455+0000", 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7049000000000001e-05 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564530+0000", 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4626e-05 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.630 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19374 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 4046848~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.076518307000001, 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049457133, 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19374 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.631 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:24.632 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:24.632 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:24.632 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:24.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.633 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.633 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515257+0000", 2024-09-16T11:44:24.633 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5740000000000001e-06 2024-09-16T11:44:24.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.633 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.633 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515262+0000", 2024-09-16T11:44:24.633 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2399999999999998e-06 2024-09-16T11:44:24.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.633 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.633 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515265+0000", 2024-09-16T11:44:24.634 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3909999999999998e-06 2024-09-16T11:44:24.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.634 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.634 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.634 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564555+0000", 2024-09-16T11:44:24.634 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049290002999999999 2024-09-16T11:44:24.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.634 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.634 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.634 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564628+0000", 2024-09-16T11:44:24.634 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2559999999999996e-05 2024-09-16T11:44:24.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564712+0000", 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4040000000000002e-05 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19375 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3301376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.076504163999999, 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049606631999999998, 2024-09-16T11:44:24.635 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.636 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.636 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.636 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.636 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.636 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19375 2024-09-16T11:44:24.636 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.636 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.636 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.636 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.636 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:24.636 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.636 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.636 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.637 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.637 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:24.637 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.637 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.637 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.637 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:24.637 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:44:24.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.637 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.637 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.637 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515271+0000", 2024-09-16T11:44:24.637 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5230000000000001e-06 2024-09-16T11:44:24.638 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.638 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.638 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.638 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515276+0000", 2024-09-16T11:44:24.638 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4650000000000002e-06 2024-09-16T11:44:24.638 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.638 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.638 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.638 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515279+0000", 2024-09-16T11:44:24.638 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0630000000000002e-06 2024-09-16T11:44:24.638 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.638 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.639 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.639 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564738+0000", 2024-09-16T11:44:24.639 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049459001000000002 2024-09-16T11:44:24.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.639 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.639 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.639 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564780+0000", 2024-09-16T11:44:24.639 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1693999999999998e-05 2024-09-16T11:44:24.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.639 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.639 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.639 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564876+0000", 2024-09-16T11:44:24.640 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5571999999999999e-05 2024-09-16T11:44:24.640 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.640 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.640 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.640 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.640 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19376 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3727360~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.640 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:24.640 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.076490251999999, 2024-09-16T11:44:24.640 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049720590000000002, 2024-09-16T11:44:24.640 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.640 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.640 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19376 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:24.641 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517103+0000", 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001820276 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517108+0000", 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5410000000000002e-06 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.642 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517120+0000", 2024-09-16T11:44:24.643 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2092e-05 2024-09-16T11:44:24.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.643 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.643 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.643 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517125+0000", 2024-09-16T11:44:24.643 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6829999999999996e-06 2024-09-16T11:44:24.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.643 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.643 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.643 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564870+0000", 2024-09-16T11:44:24.643 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047745040000000002 2024-09-16T11:44:24.644 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.644 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.644 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.644 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564921+0000", 2024-09-16T11:44:24.644 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0538999999999998e-05 2024-09-16T11:44:24.644 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.644 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.644 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.644 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565003+0000", 2024-09-16T11:44:24.644 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2418999999999998e-05 2024-09-16T11:44:24.644 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.644 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.644 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.645 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.645 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19492 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 434176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.645 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:24.645 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.074561396, 2024-09-16T11:44:24.645 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048926948999999997, 2024-09-16T11:44:24.645 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.645 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.645 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.645 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.645 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.645 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19492 2024-09-16T11:44:24.646 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.646 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.646 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.646 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.646 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:24.646 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.646 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.646 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.646 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.646 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:24.647 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.647 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.647 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.647 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.647 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:24.647 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3099999999999999e-07 2024-09-16T11:44:24.647 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.647 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.647 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.648 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517213+0000", 2024-09-16T11:44:24.648 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6599999999999994e-07 2024-09-16T11:44:24.648 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.648 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.648 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.648 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517216+0000", 2024-09-16T11:44:24.648 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.706e-06 2024-09-16T11:44:24.648 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.648 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.648 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.648 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517218+0000", 2024-09-16T11:44:24.648 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.035e-06 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565959+0000", 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048741380000000001 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566036+0000", 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7362999999999998e-05 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.649 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566139+0000", 2024-09-16T11:44:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102268 2024-09-16T11:44:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19493 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 860160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.074539725000001, 2024-09-16T11:44:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049086604999999998, 2024-09-16T11:44:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.650 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19493 2024-09-16T11:44:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.651 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.652 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.652 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.652 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:24.652 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.652 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.652 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.652 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.652 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517234+0000", 2024-09-16T11:44:24.652 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.37e-07 2024-09-16T11:44:24.652 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.652 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.652 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.652 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517236+0000", 2024-09-16T11:44:24.653 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7150000000000001e-06 2024-09-16T11:44:24.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.653 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.653 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.653 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517243+0000", 2024-09-16T11:44:24.653 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9489999999999999e-06 2024-09-16T11:44:24.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.653 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.653 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.653 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517246+0000", 2024-09-16T11:44:24.653 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.636e-06 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566163+0000", 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048917284999999998 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566237+0000", 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3196999999999997e-05 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.654 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566320+0000", 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3286000000000002e-05 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19494 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1179648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.074519109000001, 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049259562999999999, 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.655 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.656 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.656 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19494 2024-09-16T11:44:24.656 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.656 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.656 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.656 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.656 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:24.656 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.656 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.656 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.656 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.657 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:24.657 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.657 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.657 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.657 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.657 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:24.657 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:24.657 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.658 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.658 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.658 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517256+0000", 2024-09-16T11:44:24.658 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.888e-06 2024-09-16T11:44:24.658 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.658 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.658 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.658 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517265+0000", 2024-09-16T11:44:24.658 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8629999999999997e-06 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517269+0000", 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2039999999999999e-06 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566352+0000", 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049083107000000001 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.659 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566429+0000", 2024-09-16T11:44:24.660 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7025000000000002e-05 2024-09-16T11:44:24.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.660 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.660 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.660 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566514+0000", 2024-09-16T11:44:24.660 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4121000000000003e-05 2024-09-16T11:44:24.660 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.660 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.660 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.660 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.660 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19495 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1286144~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.660 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:24.661 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.074499685999999, 2024-09-16T11:44:24.661 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049492880000000003, 2024-09-16T11:44:24.661 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.661 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.661 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.661 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.661 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.661 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19495 2024-09-16T11:44:24.661 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.661 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.661 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.661 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.661 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:24.662 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.662 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.662 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.664 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:24.664 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.665 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.665 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517274+0000", 2024-09-16T11:44:24.665 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:44:24.665 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.665 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.665 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.665 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517276+0000", 2024-09-16T11:44:24.665 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8950000000000001e-06 2024-09-16T11:44:24.665 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.665 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.665 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.665 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517281+0000", 2024-09-16T11:44:24.666 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3879999999999999e-06 2024-09-16T11:44:24.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.666 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.666 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.666 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517285+0000", 2024-09-16T11:44:24.666 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5930000000000002e-06 2024-09-16T11:44:24.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.666 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.666 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.666 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566579+0000", 2024-09-16T11:44:24.666 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049294175000000003 2024-09-16T11:44:24.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.667 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.667 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.667 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566670+0000", 2024-09-16T11:44:24.667 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1096999999999999e-05 2024-09-16T11:44:24.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.667 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.667 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.667 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566766+0000", 2024-09-16T11:44:24.667 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6355999999999997e-05 2024-09-16T11:44:24.667 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.667 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.668 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.668 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.668 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19496 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1605632~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.668 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:24.668 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.074484601000002, 2024-09-16T11:44:24.668 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049632981999999999, 2024-09-16T11:44:24.668 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.669 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.669 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.669 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.669 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.669 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19496 2024-09-16T11:44:24.669 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.669 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.669 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.669 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.670 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:24.670 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.670 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.670 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.670 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.670 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:24.670 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.670 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.670 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.670 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.670 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517289+0000", 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517290+0000", 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.581e-06 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517300+0000", 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.671 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.672 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517304+0000", 2024-09-16T11:44:24.672 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:44:24.672 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.672 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.672 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.672 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566789+0000", 2024-09-16T11:44:24.672 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049485102000000003 2024-09-16T11:44:24.672 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.672 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.672 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.672 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566843+0000", 2024-09-16T11:44:24.672 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3937000000000002e-05 2024-09-16T11:44:24.672 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.673 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.673 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.673 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566921+0000", 2024-09-16T11:44:24.673 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8397000000000002e-05 2024-09-16T11:44:24.673 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.673 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.673 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.673 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.673 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.673 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19497 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 114688~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.673 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:24.673 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.074464905999999, 2024-09-16T11:44:24.673 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049761367000000001, 2024-09-16T11:44:24.674 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.674 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.674 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.674 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.674 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.674 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19497 2024-09-16T11:44:24.674 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.674 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.674 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.674 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.674 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:24.674 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.674 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.675 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.675 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.675 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:24.675 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.675 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.675 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.675 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.675 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519116+0000", 2024-09-16T11:44:24.675 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018079439999999999 2024-09-16T11:44:24.675 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.675 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.675 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.676 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519120+0000", 2024-09-16T11:44:24.676 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2760000000000002e-06 2024-09-16T11:44:24.676 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.676 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.676 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.676 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519134+0000", 2024-09-16T11:44:24.676 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3468999999999999e-05 2024-09-16T11:44:24.676 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.676 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.676 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.676 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519140+0000", 2024-09-16T11:44:24.677 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.516e-06 2024-09-16T11:44:24.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.677 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.677 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.677 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566943+0000", 2024-09-16T11:44:24.677 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047802561 2024-09-16T11:44:24.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.677 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.678 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.678 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566982+0000", 2024-09-16T11:44:24.678 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9269e-05 2024-09-16T11:44:24.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.678 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.678 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.678 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567070+0000", 2024-09-16T11:44:24.678 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7331999999999997e-05 2024-09-16T11:44:24.678 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.678 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.678 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.678 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19504 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 966656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.072536418999999, 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049045863000000002, 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19504 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.679 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.680 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:24.680 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.680 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.680 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.680 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.680 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:24.680 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.680 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.680 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.680 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.680 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:24.680 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:24.680 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.681 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.681 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.681 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519239+0000", 2024-09-16T11:44:24.681 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.962e-06 2024-09-16T11:44:24.681 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.681 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.681 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.681 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519244+0000", 2024-09-16T11:44:24.681 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5480000000000001e-06 2024-09-16T11:44:24.681 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.681 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.681 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.681 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519248+0000", 2024-09-16T11:44:24.682 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.641e-06 2024-09-16T11:44:24.682 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.682 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.682 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.682 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568150+0000", 2024-09-16T11:44:24.682 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048901864000000003 2024-09-16T11:44:24.682 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.682 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.682 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.682 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568208+0000", 2024-09-16T11:44:24.682 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8399000000000001e-05 2024-09-16T11:44:24.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.683 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.683 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.683 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568283+0000", 2024-09-16T11:44:24.683 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4105999999999998e-05 2024-09-16T11:44:24.683 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.683 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.683 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.683 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.683 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19505 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1392640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.683 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:24.683 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.072521114000001, 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049202304000000002, 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19505 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.684 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.685 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.685 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.685 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:24.685 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.685 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.685 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.685 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.685 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:24.685 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:44:24.685 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.685 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.686 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.686 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519254+0000", 2024-09-16T11:44:24.686 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.761e-06 2024-09-16T11:44:24.686 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.686 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.686 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.686 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519257+0000", 2024-09-16T11:44:24.686 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.343e-06 2024-09-16T11:44:24.686 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.686 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.686 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519259+0000", 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0559999999999999e-06 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568307+0000", 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049047445000000002 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568382+0000", 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5574999999999994e-05 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.688 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.688 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568454+0000", 2024-09-16T11:44:24.688 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1799999999999997e-05 2024-09-16T11:44:24.688 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.688 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.688 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.688 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.688 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.688 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19506 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1712128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.688 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:24.689 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.072511076000001, 2024-09-16T11:44:24.689 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049341532, 2024-09-16T11:44:24.689 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.689 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.689 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.689 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.689 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.689 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19506 2024-09-16T11:44:24.689 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.689 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.689 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521102+0000", 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018404700000000001 2024-09-16T11:44:24.690 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.691 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.691 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.691 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521106+0000", 2024-09-16T11:44:24.691 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8820000000000003e-06 2024-09-16T11:44:24.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.691 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.691 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.691 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521119+0000", 2024-09-16T11:44:24.691 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2546e-05 2024-09-16T11:44:24.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.691 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.691 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.691 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521125+0000", 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0249999999999999e-06 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568484+0000", 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047359120999999997 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568542+0000", 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8360999999999999e-05 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.692 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568604+0000", 2024-09-16T11:44:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1126999999999999e-05 2024-09-16T11:44:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19515 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1499136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.070518791000001, 2024-09-16T11:44:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048949431000000002, 2024-09-16T11:44:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.693 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19515 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.694 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523130+0000", 2024-09-16T11:44:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00187604 2024-09-16T11:44:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523134+0000", 2024-09-16T11:44:24.695 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523144+0000", 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2010000000000005e-06 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523149+0000", 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0039999999999999e-06 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.696 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570113+0000", 2024-09-16T11:44:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.046964517999999997 2024-09-16T11:44:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570157+0000", 2024-09-16T11:44:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4056e-05 2024-09-16T11:44:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570204+0000", 2024-09-16T11:44:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6591999999999997e-05 2024-09-16T11:44:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:44111 4.8 4:17f3f47e:::rbd_data.11c1e170f5f0.000000000000002f:head [delete] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 14.608538510000001, 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93903426400000001, 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 44111 2024-09-16T11:44:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7199999999999998e-07 2024-09-16T11:44:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983239+0000", 2024-09-16T11:44:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9879999999999998e-06 2024-09-16T11:44:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983248+0000", 2024-09-16T11:44:24.701 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.371e-06 2024-09-16T11:44:24.701 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.701 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.701 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:24.701 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983253+0000", 2024-09-16T11:44:24.701 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1150000000000002e-06 2024-09-16T11:44:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983303+0000", 2024-09-16T11:44:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0380000000000002e-05 2024-09-16T11:44:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:44:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983311+0000", 2024-09-16T11:44:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7519999999999996e-06 2024-09-16T11:44:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920618+0000", 2024-09-16T11:44:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93730672000000004 2024-09-16T11:44:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920730+0000", 2024-09-16T11:44:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011265900000000001 2024-09-16T11:44:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920842+0000", 2024-09-16T11:44:24.704 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111638 2024-09-16T11:44:24.704 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.704 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.704 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:24.704 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.921503+0000", 2024-09-16T11:44:24.704 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066074299999999999 2024-09-16T11:44:24.704 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.704 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922208+0000", 2024-09-16T11:44:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00070492899999999997 2024-09-16T11:44:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922233+0000", 2024-09-16T11:44:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5922999999999999e-05 2024-09-16T11:44:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922269+0000", 2024-09-16T11:44:24.707 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5373999999999997e-05 2024-09-16T11:44:24.707 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.707 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.707 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.707 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:24.707 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:24.708 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:24.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:24.710 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:24.707+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:24.809 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:24.807+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:24.848 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:24.849 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:24.849 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:24.849 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:24.857 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:24.941 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:24.939+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:24.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:44:24.976 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T11:44:24.977 INFO:teuthology.orchestra.run.smithi070.stderr:set 4.14 pg_upmap_items mapping to [6->4,1->7] 2024-09-16T11:44:25.006 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:25.006 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:25.006 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:25.006 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:25.006 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:25.010 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:25.009+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:25.016 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:25.037 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone34 from 0xe55cc2d (overlap 0xa4d3c2) to 0x0 2024-09-16T11:44:25.078 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:25.078 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:25.085 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-16T11:44:25.105 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:25.105 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:25.111 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:25.109+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:25.122 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:25.122 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:25.137 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:25.137 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:25.153 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:25.153 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:25.164 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:25.164 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:25.169 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:25.169 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:25.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:25.185 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:25.201 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:25.201 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:25.202 INFO:teuthology.orchestra.run.smithi070.stdout:5315 trunc from 0xe55cc2d to 0x49959d8 2024-09-16T11:44:25.213 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:25.211+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:25.287 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:25.288 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:25.296 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:25.313 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:25.312+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:25.394 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:25.394 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:25.402 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:25.446 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:25.444+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:25.500 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:25.509 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:25.552 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:25.551+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:25.610 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:25.610 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:25.617 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:25.615+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:25.619 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:25.718 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:25.716+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:25.756 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:25.757 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:25.765 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:25.819 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:25.818+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:25.862 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:25.862 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:25.871 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:25.921 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:25.919+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:26.011 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:26.011 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:26.020 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:26.022 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:26.019+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:26.118 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:26.118 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:26.123 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:26.121+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:26.127 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:26.224 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:26.222+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:26.225 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:26.225 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:26.234 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:26.325 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:26.324+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:26.331 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:26.331 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:26.340 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:26.426 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:26.425+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:26.479 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:26.479 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:26.487 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:26.528 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:26.527+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:26.624 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:26.624 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:26.628 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:26.627+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:26.633 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:26.729 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:26.727+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:26.772 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:26.772 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:26.781 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:26.830 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:26.829+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:26.878 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:26.878 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:26.886 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:26.931 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:26.930+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:26.986 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:26.986 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:26.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:26.995 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:44:27.040 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:27.038+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:27.106 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:27.106 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:27.116 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:27.134 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:27.132+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:27.215 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:44:27.255 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:27.256 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:27.256 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:27.254+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:27.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:27.331 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:27.332 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:27.336 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:27.334+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:27.348 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:27.349 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:27.365 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:27.365 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:27.380 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:27.380 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:27.396 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:27.396 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:27.409 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:9081 4.b 4:d317bc5b:::rbd_data.11c1dc795e15.000000000000000d:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-16T11:44:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:44:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 41.352572424999998, 2024-09-16T11:44:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027796406999999999, 2024-09-16T11:44:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 9081 2024-09-16T11:44:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:44:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053956+0000", 2024-09-16T11:44:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053957+0000", 2024-09-16T11:44:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7010000000000001e-06 2024-09-16T11:44:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053965+0000", 2024-09-16T11:44:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5939999999999996e-06 2024-09-16T11:44:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053980+0000", 2024-09-16T11:44:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5099000000000001e-05 2024-09-16T11:44:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.053986+0000", 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5759999999999996e-06 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.054023+0000", 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999999e-05 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.054076+0000", 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2852999999999997e-05 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.058880+0000", 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048042639999999999 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081663+0000", 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022782908000000001 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081672+0000", 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0440000000000001e-06 2024-09-16T11:44:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081707+0000", 2024-09-16T11:44:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4986e-05 2024-09-16T11:44:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:46.081752+0000", 2024-09-16T11:44:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4781999999999999e-05 2024-09-16T11:44:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:45905 4.1e 4.248d413e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.289465+0000", 2024-09-16T11:44:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 16.117062938, 2024-09-16T11:44:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024768528000000001, 2024-09-16T11:44:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 45905 2024-09-16T11:44:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289465+0000", 2024-09-16T11:44:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289465+0000", 2024-09-16T11:44:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289466+0000", 2024-09-16T11:44:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0900000000000002e-07 2024-09-16T11:44:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289481+0000", 2024-09-16T11:44:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5291999999999999e-05 2024-09-16T11:44:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289488+0000", 2024-09-16T11:44:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8379999999999996e-06 2024-09-16T11:44:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.289491+0000", 2024-09-16T11:44:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6150000000000001e-06 2024-09-16T11:44:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.314228+0000", 2024-09-16T11:44:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024736719000000001 2024-09-16T11:44:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.314234+0000", 2024-09-16T11:44:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5550000000000002e-06 2024-09-16T11:44:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:46788 4.16 4:6ec64c8a:::rbd_data.11c1e521eb15.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 8060928] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T11:44:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:11.326728+0000", 2024-09-16T11:44:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 16.079800117000001, 2024-09-16T11:44:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028859883999999999, 2024-09-16T11:44:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 46788 2024-09-16T11:44:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326728+0000", 2024-09-16T11:44:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326728+0000", 2024-09-16T11:44:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.326728+0000", 2024-09-16T11:44:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327570+0000", 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00084174300000000005 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327582+0000", 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2435e-05 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.327589+0000", 2024-09-16T11:44:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1970000000000001e-06 2024-09-16T11:44:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349312+0000", 2024-09-16T11:44:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.021723524000000001 2024-09-16T11:44:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.349465+0000", 2024-09-16T11:44:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015305200000000001 2024-09-16T11:44:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3", 2024-09-16T11:44:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.350219+0000", 2024-09-16T11:44:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00075365399999999998 2024-09-16T11:44:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.355296+0000", 2024-09-16T11:44:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0050773210000000001 2024-09-16T11:44:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.355469+0000", 2024-09-16T11:44:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017311400000000001 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.355552+0000", 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3018999999999999e-05 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:11.355588+0000", 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5468e-05 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55262 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 2818048~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209313+0000", 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1972150749999999, 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.023066109000000001, 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55262 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209313+0000", 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209313+0000", 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209313+0000", 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3799999999999998e-07 2024-09-16T11:44:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209318+0000", 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.023e-06 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209327+0000", 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.67e-06 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209331+0000", 2024-09-16T11:44:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7249999999999999e-06 2024-09-16T11:44:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.232187+0000", 2024-09-16T11:44:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.022856385999999999 2024-09-16T11:44:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.232263+0000", 2024-09-16T11:44:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6318999999999999e-05 2024-09-16T11:44:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.232379+0000", 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000115648 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55263 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 3014656~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209335+0000", 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1971933379999999, 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.023273572999999999, 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55263 2024-09-16T11:44:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209335+0000", 2024-09-16T11:44:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209335+0000", 2024-09-16T11:44:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209335+0000", 2024-09-16T11:44:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:44:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209337+0000", 2024-09-16T11:44:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0490000000000002e-06 2024-09-16T11:44:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209343+0000", 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9610000000000002e-06 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209346+0000", 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1209999999999998e-06 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.232422+0000", 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.023075559999999998 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.232497+0000", 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5203999999999994e-05 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.232608+0000", 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111297 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55264 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209350+0000", 2024-09-16T11:44:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1971779699999998, 2024-09-16T11:44:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.023468060999999998, 2024-09-16T11:44:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55264 2024-09-16T11:44:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209350+0000", 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209350+0000", 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209350+0000", 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209352+0000", 2024-09-16T11:44:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.719e-06 2024-09-16T11:44:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209357+0000", 2024-09-16T11:44:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2560000000000002e-06 2024-09-16T11:44:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209361+0000", 2024-09-16T11:44:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1669999999999999e-06 2024-09-16T11:44:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.232649+0000", 2024-09-16T11:44:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.023288876 2024-09-16T11:44:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.232726+0000", 2024-09-16T11:44:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6525000000000003e-05 2024-09-16T11:44:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.232818+0000", 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2207000000000007e-05 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55265 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209364+0000", 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1971639500000002, 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.023668301999999999, 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55265 2024-09-16T11:44:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209364+0000", 2024-09-16T11:44:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209364+0000", 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209364+0000", 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209366+0000", 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9190000000000002e-06 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209372+0000", 2024-09-16T11:44:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7320000000000003e-06 2024-09-16T11:44:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209375+0000", 2024-09-16T11:44:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3280000000000002e-06 2024-09-16T11:44:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.232859+0000", 2024-09-16T11:44:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.023483898999999999 2024-09-16T11:44:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.232937+0000", 2024-09-16T11:44:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7621999999999996e-05 2024-09-16T11:44:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233032+0000", 2024-09-16T11:44:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5451999999999993e-05 2024-09-16T11:44:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55266 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209379+0000", 2024-09-16T11:44:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1971490189999998, 2024-09-16T11:44:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0237868, 2024-09-16T11:44:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55266 2024-09-16T11:44:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209379+0000", 2024-09-16T11:44:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209379+0000", 2024-09-16T11:44:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209379+0000", 2024-09-16T11:44:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209381+0000", 2024-09-16T11:44:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.683e-06 2024-09-16T11:44:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209386+0000", 2024-09-16T11:44:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.429e-06 2024-09-16T11:44:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209390+0000", 2024-09-16T11:44:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1590000000000002e-06 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233068+0000", 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.023678092000000001 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233116+0000", 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8029999999999999e-05 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233166+0000", 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0067e-05 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55267 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209393+0000", 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1971351270000001, 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.023900293, 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55267 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209393+0000", 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209393+0000", 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209393+0000", 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209395+0000", 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209400+0000", 2024-09-16T11:44:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1869999999999996e-06 2024-09-16T11:44:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209403+0000", 2024-09-16T11:44:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2200000000000001e-06 2024-09-16T11:44:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233194+0000", 2024-09-16T11:44:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.02379069 2024-09-16T11:44:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233243+0000", 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9098999999999999e-05 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233293+0000", 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0118999999999999e-05 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55268 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209407+0000", 2024-09-16T11:44:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1971213600000001, 2024-09-16T11:44:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024099980999999999, 2024-09-16T11:44:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55268 2024-09-16T11:44:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209407+0000", 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209407+0000", 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209407+0000", 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1800000000000002e-07 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209409+0000", 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.624e-06 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209414+0000", 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5010000000000004e-06 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209417+0000", 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2480000000000001e-06 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233332+0000", 2024-09-16T11:44:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.023914582 2024-09-16T11:44:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233412+0000", 2024-09-16T11:44:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9667999999999997e-05 2024-09-16T11:44:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233507+0000", 2024-09-16T11:44:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5039999999999998e-05 2024-09-16T11:44:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55269 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 4390912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209421+0000", 2024-09-16T11:44:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.197107387, 2024-09-16T11:44:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024196359000000001, 2024-09-16T11:44:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55269 2024-09-16T11:44:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209421+0000", 2024-09-16T11:44:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209421+0000", 2024-09-16T11:44:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209421+0000", 2024-09-16T11:44:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:44:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209423+0000", 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.686e-06 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209428+0000", 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0180000000000001e-06 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209431+0000", 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0920000000000002e-06 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233536+0000", 2024-09-16T11:44:27.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024105237000000002 2024-09-16T11:44:27.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233571+0000", 2024-09-16T11:44:27.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4650999999999997e-05 2024-09-16T11:44:27.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233617+0000", 2024-09-16T11:44:27.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6400000000000003e-05 2024-09-16T11:44:27.465 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.465 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.465 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.465 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55270 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 4587520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.465 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209434+0000", 2024-09-16T11:44:27.465 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1970937020000001, 2024-09-16T11:44:27.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024383986, 2024-09-16T11:44:27.465 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55270 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209434+0000", 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209434+0000", 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209435+0000", 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.84e-07 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209436+0000", 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.658e-06 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209441+0000", 2024-09-16T11:44:27.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0699999999999997e-06 2024-09-16T11:44:27.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209444+0000", 2024-09-16T11:44:27.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0529999999999998e-06 2024-09-16T11:44:27.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233652+0000", 2024-09-16T11:44:27.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024207620999999999 2024-09-16T11:44:27.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233726+0000", 2024-09-16T11:44:27.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4258000000000006e-05 2024-09-16T11:44:27.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233818+0000", 2024-09-16T11:44:27.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1941999999999995e-05 2024-09-16T11:44:27.469 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.469 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.470 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55271 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 4849664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.470 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209448+0000", 2024-09-16T11:44:27.470 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1970803820000002, 2024-09-16T11:44:27.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024477663, 2024-09-16T11:44:27.470 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.470 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.470 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.470 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.470 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.470 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55271 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209448+0000", 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209448+0000", 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209448+0000", 2024-09-16T11:44:27.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6199999999999999e-07 2024-09-16T11:44:27.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209449+0000", 2024-09-16T11:44:27.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.548e-06 2024-09-16T11:44:27.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209454+0000", 2024-09-16T11:44:27.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.011e-06 2024-09-16T11:44:27.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209457+0000", 2024-09-16T11:44:27.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.994e-06 2024-09-16T11:44:27.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233846+0000", 2024-09-16T11:44:27.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024388546000000001 2024-09-16T11:44:27.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233879+0000", 2024-09-16T11:44:27.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3454000000000002e-05 2024-09-16T11:44:27.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233925+0000", 2024-09-16T11:44:27.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5847999999999999e-05 2024-09-16T11:44:27.474 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.474 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.474 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.475 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55272 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 5046272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.475 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209461+0000", 2024-09-16T11:44:27.475 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1970669809999999, 2024-09-16T11:44:27.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024754264000000002, 2024-09-16T11:44:27.475 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.475 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.475 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.475 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.475 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.475 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55272 2024-09-16T11:44:27.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.475 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209461+0000", 2024-09-16T11:44:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209461+0000", 2024-09-16T11:44:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209461+0000", 2024-09-16T11:44:27.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2300000000000002e-07 2024-09-16T11:44:27.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209463+0000", 2024-09-16T11:44:27.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7290000000000001e-06 2024-09-16T11:44:27.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209469+0000", 2024-09-16T11:44:27.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4280000000000004e-06 2024-09-16T11:44:27.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209472+0000", 2024-09-16T11:44:27.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0510000000000001e-06 2024-09-16T11:44:27.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.233961+0000", 2024-09-16T11:44:27.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024489667999999999 2024-09-16T11:44:27.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234036+0000", 2024-09-16T11:44:27.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5122000000000003e-05 2024-09-16T11:44:27.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234215+0000", 2024-09-16T11:44:27.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017894300000000001 2024-09-16T11:44:27.479 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.479 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.479 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.479 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55273 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.480 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209475+0000", 2024-09-16T11:44:27.480 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.197052818, 2024-09-16T11:44:27.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024843212999999999, 2024-09-16T11:44:27.480 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.480 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.480 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.480 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.480 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.480 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55273 2024-09-16T11:44:27.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.480 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209475+0000", 2024-09-16T11:44:27.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209475+0000", 2024-09-16T11:44:27.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209476+0000", 2024-09-16T11:44:27.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9700000000000003e-07 2024-09-16T11:44:27.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209477+0000", 2024-09-16T11:44:27.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.841e-06 2024-09-16T11:44:27.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209483+0000", 2024-09-16T11:44:27.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5500000000000002e-06 2024-09-16T11:44:27.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209486+0000", 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0970000000000002e-06 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234236+0000", 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024749653999999999 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234271+0000", 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5846999999999997e-05 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234318+0000", 2024-09-16T11:44:27.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6927e-05 2024-09-16T11:44:27.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.484 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.484 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55274 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.484 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209489+0000", 2024-09-16T11:44:27.484 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1970386140000002, 2024-09-16T11:44:27.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.025030489999999999, 2024-09-16T11:44:27.485 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.485 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.485 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55274 2024-09-16T11:44:27.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.485 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209489+0000", 2024-09-16T11:44:27.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209489+0000", 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209490+0000", 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2599999999999998e-07 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209496+0000", 2024-09-16T11:44:27.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:44:27.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209504+0000", 2024-09-16T11:44:27.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0989999999999999e-06 2024-09-16T11:44:27.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209507+0000", 2024-09-16T11:44:27.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7050000000000001e-06 2024-09-16T11:44:27.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234355+0000", 2024-09-16T11:44:27.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024847656999999999 2024-09-16T11:44:27.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234428+0000", 2024-09-16T11:44:27.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3266e-05 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234520+0000", 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1551000000000005e-05 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55275 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 5701632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209511+0000", 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1970166309999999, 2024-09-16T11:44:27.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.025116764, 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55275 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209511+0000", 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209511+0000", 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209512+0000", 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.32e-07 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209514+0000", 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.04e-06 2024-09-16T11:44:27.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209520+0000", 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8989999999999998e-06 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209523+0000", 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4460000000000001e-06 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234548+0000", 2024-09-16T11:44:27.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.025024956000000001 2024-09-16T11:44:27.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234584+0000", 2024-09-16T11:44:27.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5800000000000003e-05 2024-09-16T11:44:27.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234628+0000", 2024-09-16T11:44:27.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4190999999999998e-05 2024-09-16T11:44:27.493 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.493 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.493 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55276 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 5963776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209527+0000", 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1970006789999998, 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.025305503, 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55276 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.494 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209527+0000", 2024-09-16T11:44:27.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209527+0000", 2024-09-16T11:44:27.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209528+0000", 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3599999999999999e-07 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209530+0000", 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9580000000000001e-06 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209535+0000", 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.434e-06 2024-09-16T11:44:27.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209538+0000", 2024-09-16T11:44:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.202e-06 2024-09-16T11:44:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234664+0000", 2024-09-16T11:44:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.025125345 2024-09-16T11:44:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234740+0000", 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6231999999999998e-05 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234833+0000", 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2996000000000002e-05 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55277 4.1e 4:78a4531b:::rbd_data.11c1b63a8930.0000000000000007:25 [sparse-read 6160384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209542+0000", 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.196985862, 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.025389161, 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55277 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209542+0000", 2024-09-16T11:44:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209542+0000", 2024-09-16T11:44:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209542+0000", 2024-09-16T11:44:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:44:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209544+0000", 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5910000000000001e-06 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209549+0000", 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2480000000000004e-06 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209552+0000", 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-06 2024-09-16T11:44:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234848+0000", 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.025295449000000001 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234887+0000", 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8625999999999999e-05 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234931+0000", 2024-09-16T11:44:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4818999999999999e-05 2024-09-16T11:44:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55278 4.6 4:6509fb4d:::rbd_data.11c11a49c2da.0000000000000002:head [stat,truncate 3593154] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T11:44:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:24.209556+0000", 2024-09-16T11:44:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 3.1969717850000001, 2024-09-16T11:44:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027415042000000001, 2024-09-16T11:44:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55278 2024-09-16T11:44:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209556+0000", 2024-09-16T11:44:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209556+0000", 2024-09-16T11:44:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209557+0000", 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9499999999999998e-07 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209558+0000", 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9589999999999998e-06 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.209565+0000", 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0159999999999997e-06 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.210659+0000", 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010941690000000001 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234870+0000", 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.024211759999999999 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.234937+0000", 2024-09-16T11:44:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6247000000000005e-05 2024-09-16T11:44:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T11:44:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.235079+0000", 2024-09-16T11:44:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000141948 2024-09-16T11:44:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.236661+0000", 2024-09-16T11:44:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001582162 2024-09-16T11:44:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.236909+0000", 2024-09-16T11:44:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024802500000000002 2024-09-16T11:44:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.236940+0000", 2024-09-16T11:44:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.167e-05 2024-09-16T11:44:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:24.236971+0000", 2024-09-16T11:44:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0790999999999997e-05 2024-09-16T11:44:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.510 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:27.510 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:27.511 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:27.512 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:27.512 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:27.512 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:27.513 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:27.513 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:27.514 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:27.509+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:27.612 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:27.611+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:27.650 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:27.650 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:27.650 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:27.650 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:27.659 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:27.685 INFO:teuthology.orchestra.run.smithi070.stdout:5317 write 0xcf32f39 thru 0xcf3c833 (0x98fb bytes) 2024-09-16T11:44:27.689 INFO:teuthology.orchestra.run.smithi070.stdout:5319 writesame 0xa06ffb6 thru 0xa076556 data_size 0x1(0x65a1 bytes) 2024-09-16T11:44:27.703 INFO:teuthology.orchestra.run.smithi070.stdout:5320 read 0x2166fb9 thru 0x217589b (0xe8e3 bytes) 2024-09-16T11:44:27.704 INFO:teuthology.orchestra.run.smithi070.stdout:5322 read 0x4d8f0b5 thru 0x4d9335a (0x42a6 bytes) 2024-09-16T11:44:27.704 INFO:teuthology.orchestra.run.smithi070.stdout:5323 read 0x91fb99f thru 0x91fba63 (0xc5 bytes) 2024-09-16T11:44:27.704 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:27.704 INFO:teuthology.orchestra.run.smithi070.stdout:5325 write 0x6bd11c5 thru 0x6be03b3 (0xf1ef bytes) 2024-09-16T11:44:27.707 INFO:teuthology.orchestra.run.smithi070.stdout:5327 read 0x83b204e thru 0x83bd2d9 (0xb28c bytes) 2024-09-16T11:44:27.707 INFO:teuthology.orchestra.run.smithi070.stdout:5328 writesame 0x1ca7b5b thru 0x1cb6c97 data_size 0x1(0xf13d bytes) 2024-09-16T11:44:27.713 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:27.712+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:27.739 INFO:teuthology.orchestra.run.smithi070.stdout:5329 write 0x3648e17 thru 0x3654ab9 (0xbca3 bytes) 2024-09-16T11:44:27.744 INFO:teuthology.orchestra.run.smithi070.stdout:5330 punch from 0x34cea16 to 0x34d2a1e, (0x4008 bytes) 2024-09-16T11:44:27.744 INFO:teuthology.orchestra.run.smithi070.stdout:5331 writesame 0xaa5879 thru 0xab356c data_size 0x4(0xdcf4 bytes) 2024-09-16T11:44:27.754 INFO:teuthology.orchestra.run.smithi070.stdout:5333 punch from 0x89af to 0x168e4, (0xdf35 bytes) 2024-09-16T11:44:27.771 INFO:teuthology.orchestra.run.smithi070.stdout:5334 trunc from 0xcf3c834 to 0xa7c2dc 2024-09-16T11:44:27.788 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:27.789 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:27.789 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:27.789 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:27.789 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:27.801 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:27.802 INFO:teuthology.orchestra.run.smithi070.stdout:5335 read 0x194add thru 0x197d31 (0x3255 bytes) 2024-09-16T11:44:27.802 INFO:teuthology.orchestra.run.smithi070.stdout:5338 read 0x37fba0 thru 0x383b41 (0x3fa2 bytes) 2024-09-16T11:44:27.802 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:27.802 INFO:teuthology.orchestra.run.smithi070.stdout:5340 read 0x94b463 thru 0x953a5d (0x85fb bytes) 2024-09-16T11:44:27.811 INFO:teuthology.orchestra.run.smithi070.stdout:5341 write 0x4d5558f thru 0x4d5e8cc (0x933e bytes) 2024-09-16T11:44:27.814 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:27.812+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:27.822 INFO:teuthology.orchestra.run.smithi070.stdout:5342 writesame 0x77626da thru 0x7768da4 data_size 0x13(0x66cb bytes) 2024-09-16T11:44:27.830 INFO:teuthology.orchestra.run.smithi070.stdout:5343 write 0xa113bc1 thru 0xa11a3d3 (0x6813 bytes) 2024-09-16T11:44:27.843 INFO:teuthology.orchestra.run.smithi070.stdout:5344 trunc from 0xa11a3d4 to 0x1f1b09d 2024-09-16T11:44:27.864 INFO:teuthology.orchestra.run.smithi070.stdout:5345 write 0x764fece thru 0x765933d (0x9470 bytes) 2024-09-16T11:44:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:5346 read 0x5817819 thru 0x581edb2 (0x759a bytes) 2024-09-16T11:44:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:5347 punch from 0x67143e4 to 0x6717b1c, (0x3738 bytes) 2024-09-16T11:44:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:5349 punch from 0x37bf67d to 0x37cbb52, (0xc4d5 bytes) 2024-09-16T11:44:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:27.868 INFO:teuthology.orchestra.run.smithi070.stdout:5351 trunc from 0x765933e to 0x666a7e3 2024-09-16T11:44:27.877 INFO:teuthology.orchestra.run.smithi070.stdout:5353 read 0x11f7cf5 thru 0x1202b35 (0xae41 bytes) 2024-09-16T11:44:27.877 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:27.883 INFO:teuthology.orchestra.run.smithi070.stdout:5358 write 0x7745d22 thru 0x774a8ec (0x4bcb bytes) 2024-09-16T11:44:27.894 INFO:teuthology.orchestra.run.smithi070.stdout:5360 writesame 0xa0c4a9d thru 0xa0ca2cb data_size 0x81(0x582f bytes) 2024-09-16T11:44:27.894 INFO:teuthology.orchestra.run.smithi070.stdout:5361 write 0x97aa826 thru 0x97ae8c0 (0x409b bytes) 2024-09-16T11:44:27.906 INFO:teuthology.orchestra.run.smithi070.stdout:5362 trunc from 0xa0ca2cc to 0x2ac595d 2024-09-16T11:44:27.915 INFO:teuthology.orchestra.run.smithi070.stdout:5363 read 0x20fb507 thru 0x21072a5 (0xbd9f bytes) 2024-09-16T11:44:27.915 INFO:teuthology.orchestra.run.smithi070.stdout:5365 read 0xd6c6 thru 0xfc30 (0x256b bytes) 2024-09-16T11:44:27.916 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:27.914+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:27.920 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:27.920 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:27.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:27.920 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:27.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.920 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19364 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1171456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.404261996000002, 2024-09-16T11:44:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049001805000000002, 2024-09-16T11:44:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19364 2024-09-16T11:44:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513280+0000", 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7090000000000001e-06 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513286+0000", 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5840000000000002e-06 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513290+0000", 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3680000000000004e-06 2024-09-16T11:44:27.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562121+0000", 2024-09-16T11:44:27.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048831324000000002 2024-09-16T11:44:27.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562195+0000", 2024-09-16T11:44:27.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3415000000000001e-05 2024-09-16T11:44:27.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562280+0000", 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4975999999999994e-05 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19365 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1490944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.404242257, 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049335349000000001, 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.925 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.926 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.926 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19365 2024-09-16T11:44:27.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.926 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:27.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:27.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513300+0000", 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3020000000000002e-06 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513306+0000", 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8780000000000003e-06 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513310+0000", 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1309999999999999e-06 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562322+0000", 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049011635999999997 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562544+0000", 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022168000000000001 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562633+0000", 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9350000000000003e-05 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19366 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1916928~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.929 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.404225324999999, 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049865073000000003, 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19366 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:27.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515102+0000", 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017877419999999999 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515106+0000", 2024-09-16T11:44:27.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4450000000000001e-06 2024-09-16T11:44:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515113+0000", 2024-09-16T11:44:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7680000000000008e-06 2024-09-16T11:44:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515117+0000", 2024-09-16T11:44:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:44:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562756+0000", 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047638579 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563093+0000", 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000337516 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563180+0000", 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6420000000000003e-05 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.933 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.934 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19371 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3194880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.934 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:27.935 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.402328118, 2024-09-16T11:44:27.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048948356999999998, 2024-09-16T11:44:27.935 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.935 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.935 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.935 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.935 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.935 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19371 2024-09-16T11:44:27.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.935 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:27.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:27.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:27.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:27.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515214+0000", 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5969999999999999e-06 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515219+0000", 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4559999999999999e-06 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515223+0000", 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:44:27.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563991+0000", 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048768911999999998 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564063+0000", 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2010999999999999e-05 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564160+0000", 2024-09-16T11:44:27.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6617e-05 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19372 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3620864~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.402313900999999, 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049130502, 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.939 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.940 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19372 2024-09-16T11:44:27.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.940 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:27.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:27.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515228+0000", 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515233+0000", 2024-09-16T11:44:27.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515237+0000", 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.27e-06 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564199+0000", 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048961787999999999 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564275+0000", 2024-09-16T11:44:27.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6001000000000006e-05 2024-09-16T11:44:27.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564356+0000", 2024-09-16T11:44:27.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1941999999999996e-05 2024-09-16T11:44:27.943 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.943 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.943 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.943 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19373 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3940352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.943 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:27.944 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.402299501000002, 2024-09-16T11:44:27.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049289588000000002, 2024-09-16T11:44:27.944 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.944 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.944 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.944 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.944 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.944 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19373 2024-09-16T11:44:27.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.944 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515241+0000", 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515242+0000", 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.528e-06 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515248+0000", 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515251+0000", 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2480000000000001e-06 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564388+0000", 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049137313000000002 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564455+0000", 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7049000000000001e-05 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564530+0000", 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4626e-05 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.947 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19374 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 4046848~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.402285098, 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049457133, 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19374 2024-09-16T11:44:27.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515257+0000", 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5740000000000001e-06 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515262+0000", 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2399999999999998e-06 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515265+0000", 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3909999999999998e-06 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564555+0000", 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049290002999999999 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564628+0000", 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2559999999999996e-05 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.952 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564712+0000", 2024-09-16T11:44:27.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4040000000000002e-05 2024-09-16T11:44:27.952 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.952 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.952 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.952 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19375 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3301376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.952 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:27.952 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.402270954999999, 2024-09-16T11:44:27.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049606631999999998, 2024-09-16T11:44:27.952 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.952 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19375 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.953 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:27.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:27.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:44:27.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515271+0000", 2024-09-16T11:44:27.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5230000000000001e-06 2024-09-16T11:44:27.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515276+0000", 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4650000000000002e-06 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515279+0000", 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0630000000000002e-06 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564738+0000", 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049459001000000002 2024-09-16T11:44:27.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564780+0000", 2024-09-16T11:44:27.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1693999999999998e-05 2024-09-16T11:44:27.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564876+0000", 2024-09-16T11:44:27.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5571999999999999e-05 2024-09-16T11:44:27.956 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.956 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.956 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19376 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3727360~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.402257042999999, 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049720590000000002, 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19376 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517103+0000", 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001820276 2024-09-16T11:44:27.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517108+0000", 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5410000000000002e-06 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517120+0000", 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2092e-05 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517125+0000", 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6829999999999996e-06 2024-09-16T11:44:27.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564870+0000", 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047745040000000002 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564921+0000", 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0538999999999998e-05 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565003+0000", 2024-09-16T11:44:27.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2418999999999998e-05 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19492 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 434176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.400328187, 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048926948999999997, 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.961 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19492 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3099999999999999e-07 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517213+0000", 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6599999999999994e-07 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517216+0000", 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.706e-06 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517218+0000", 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.035e-06 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565959+0000", 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048741380000000001 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566036+0000", 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7362999999999998e-05 2024-09-16T11:44:27.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566139+0000", 2024-09-16T11:44:27.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102268 2024-09-16T11:44:27.965 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.965 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.965 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.965 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19493 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 860160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.965 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:27.965 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.400306516000001, 2024-09-16T11:44:27.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049086604999999998, 2024-09-16T11:44:27.969 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.969 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.969 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.969 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.969 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.969 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19493 2024-09-16T11:44:27.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.969 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:27.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:27.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517234+0000", 2024-09-16T11:44:27.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.37e-07 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517236+0000", 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7150000000000001e-06 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517243+0000", 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9489999999999999e-06 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517246+0000", 2024-09-16T11:44:27.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.636e-06 2024-09-16T11:44:27.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566163+0000", 2024-09-16T11:44:27.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048917284999999998 2024-09-16T11:44:27.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566237+0000", 2024-09-16T11:44:27.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3196999999999997e-05 2024-09-16T11:44:27.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566320+0000", 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3286000000000002e-05 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19494 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1179648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.4002859, 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049259562999999999, 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.973 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19494 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:27.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:27.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:27.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517256+0000", 2024-09-16T11:44:27.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.888e-06 2024-09-16T11:44:27.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517265+0000", 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8629999999999997e-06 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517269+0000", 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2039999999999999e-06 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566352+0000", 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049083107000000001 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566429+0000", 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7025000000000002e-05 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566514+0000", 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4121000000000003e-05 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19495 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1286144~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.977 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.400266477000002, 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049492880000000003, 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19495 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:27.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517274+0000", 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517276+0000", 2024-09-16T11:44:27.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8950000000000001e-06 2024-09-16T11:44:27.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517281+0000", 2024-09-16T11:44:27.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3879999999999999e-06 2024-09-16T11:44:27.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517285+0000", 2024-09-16T11:44:27.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5930000000000002e-06 2024-09-16T11:44:27.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566579+0000", 2024-09-16T11:44:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049294175000000003 2024-09-16T11:44:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566670+0000", 2024-09-16T11:44:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1096999999999999e-05 2024-09-16T11:44:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566766+0000", 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6355999999999997e-05 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19496 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1605632~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.400251392000001, 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049632981999999999, 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.982 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19496 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517289+0000", 2024-09-16T11:44:27.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:27.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517290+0000", 2024-09-16T11:44:27.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.581e-06 2024-09-16T11:44:27.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517300+0000", 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517304+0000", 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566789+0000", 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049485102000000003 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566843+0000", 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3937000000000002e-05 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566921+0000", 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8397000000000002e-05 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19497 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 114688~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.986 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:27.987 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.400231697000002, 2024-09-16T11:44:27.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049761367000000001, 2024-09-16T11:44:27.987 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.987 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.987 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.987 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.987 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.987 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19497 2024-09-16T11:44:27.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.987 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.987 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.987 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519116+0000", 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018079439999999999 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519120+0000", 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2760000000000002e-06 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519134+0000", 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3468999999999999e-05 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519140+0000", 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.516e-06 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566943+0000", 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047802561 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566982+0000", 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9269e-05 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567070+0000", 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7331999999999997e-05 2024-09-16T11:44:27.990 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19504 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 966656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.398303210000002, 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049045863000000002, 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19504 2024-09-16T11:44:27.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:27.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519239+0000", 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.962e-06 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519244+0000", 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5480000000000001e-06 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519248+0000", 2024-09-16T11:44:27.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.641e-06 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568150+0000", 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048901864000000003 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568208+0000", 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8399000000000001e-05 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568283+0000", 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4105999999999998e-05 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19505 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1392640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.398287904999997, 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049202304000000002, 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:27.995 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19505 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:27.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:27.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:27.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:44:27.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:27.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519254+0000", 2024-09-16T11:44:27.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.761e-06 2024-09-16T11:44:27.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:27.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519257+0000", 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.343e-06 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519259+0000", 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0559999999999999e-06 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568307+0000", 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049047445000000002 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568382+0000", 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5574999999999994e-05 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568454+0000", 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1799999999999997e-05 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19506 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1712128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:27.999 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.398277866999997, 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049341532, 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19506 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521102+0000", 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018404700000000001 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521106+0000", 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8820000000000003e-06 2024-09-16T11:44:28.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521119+0000", 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2546e-05 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521125+0000", 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0249999999999999e-06 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568484+0000", 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047359120999999997 2024-09-16T11:44:28.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568542+0000", 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8360999999999999e-05 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568604+0000", 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1126999999999999e-05 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19515 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1499136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.396285581999997, 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048949431000000002, 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19515 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523130+0000", 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00187604 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523134+0000", 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523144+0000", 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2010000000000005e-06 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523149+0000", 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0039999999999999e-06 2024-09-16T11:44:28.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570113+0000", 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.046964517999999997 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570157+0000", 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4056e-05 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570204+0000", 2024-09-16T11:44:28.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6591999999999997e-05 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:44111 4.8 4:17f3f47e:::rbd_data.11c1e170f5f0.000000000000002f:head [delete] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.934305300999998, 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93903426400000001, 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.008 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 44111 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7199999999999998e-07 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983239+0000", 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9879999999999998e-06 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983248+0000", 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.371e-06 2024-09-16T11:44:28.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983253+0000", 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1150000000000002e-06 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983303+0000", 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0380000000000002e-05 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983311+0000", 2024-09-16T11:44:28.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7519999999999996e-06 2024-09-16T11:44:28.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.012 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920618+0000", 2024-09-16T11:44:28.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93730672000000004 2024-09-16T11:44:28.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.012 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920730+0000", 2024-09-16T11:44:28.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011265900000000001 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920842+0000", 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111638 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.921503+0000", 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066074299999999999 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922208+0000", 2024-09-16T11:44:28.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00070492899999999997 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922233+0000", 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5922999999999999e-05 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922269+0000", 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5373999999999997e-05 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.014 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.015 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.015 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:28.015 INFO:teuthology.orchestra.run.smithi070.stdout:5366 write 0x6b0fe52 thru 0x6b1aa56 (0xac05 bytes) 2024-09-16T11:44:28.016 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:28.018 INFO:teuthology.orchestra.run.smithi070.stdout:5367 writesame 0xc89a3dd thru 0xc8a9357 data_size 0x1(0xef7b bytes) 2024-09-16T11:44:28.018 INFO:teuthology.orchestra.run.smithi070.stdout:5369 punch from 0x61a7561 to 0x61b00c5, (0x8b64 bytes) 2024-09-16T11:44:28.018 INFO:teuthology.orchestra.run.smithi070.stdout:5370 punch from 0x3edcef2 to 0x3ee6d1e, (0x9e2c bytes) 2024-09-16T11:44:28.018 INFO:teuthology.orchestra.run.smithi070.stdout:5371 punch from 0xd798bb to 0xd7ff24, (0x6669 bytes) 2024-09-16T11:44:28.018 INFO:teuthology.orchestra.run.smithi070.stdout:5372 write 0x2585eba thru 0x258f7a9 (0x98f0 bytes) 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:5374 read 0x8c2a5f thru 0x8c4b5d (0x20ff bytes) 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:5376 write 0x5006ae thru 0x5016e4 (0x1037 bytes) 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:5377 read 0x5b3eea6 thru 0x5b4e989 (0xfae4 bytes) 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:5378 write 0x2fe0d6f thru 0x2fe8ca2 (0x7f34 bytes) 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:5380 write 0x876ba61 thru 0x8775e82 (0xa422 bytes) 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:5382 punch from 0x72ae090 to 0x72ba5df, (0xc54f bytes) 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:5383 trunc from 0xc8a9358 to 0x5e7d186 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:5384 read 0x281d3dd thru 0x2821f26 (0x4b4a bytes) 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:5387 read 0x4c9d98d thru 0x4c9dfe9 (0x65d bytes) 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:5391 punch from 0x3e5ab3 to 0x3f2ac2, (0xd00f bytes) 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:5395 writesame 0xb05a869 thru 0xb068589 data_size 0x1(0xdd21 bytes) 2024-09-16T11:44:28.019 INFO:teuthology.orchestra.run.smithi070.stdout:5396 writesame 0x3ce5ebf thru 0x3cefd44 data_size 0x67(0x9e86 bytes) 2024-09-16T11:44:28.020 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:28.020 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T11:44:28.020 INFO:teuthology.orchestra.run.smithi070.stdout:5400 writesame 0xc90006b thru 0xc907ed9 data_size 0x1(0x7e6f bytes) 2024-09-16T11:44:28.020 INFO:teuthology.orchestra.run.smithi070.stdout:5404 trunc from 0xc907eda to 0x7f33423 2024-09-16T11:44:28.021 INFO:teuthology.orchestra.run.smithi070.stdout:5405 write 0xb3db192 thru 0xb3e0e2c (0x5c9b bytes) 2024-09-16T11:44:28.021 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:28.014+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:28.022 INFO:teuthology.orchestra.run.smithi070.stdout:5406 trunc from 0xb3e0e2d to 0x6658baf 2024-09-16T11:44:28.029 INFO:teuthology.orchestra.run.smithi070.stdout:5407 punch from 0x6ff8cb to 0x7021e7, (0x291c bytes) 2024-09-16T11:44:28.029 INFO:teuthology.orchestra.run.smithi070.stdout:5408 read 0x490bdf9 thru 0x49185ff (0xc807 bytes) 2024-09-16T11:44:28.039 INFO:teuthology.orchestra.run.smithi070.stdout:5409 trunc from 0x6658baf to 0xed42f6f 2024-09-16T11:44:28.044 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:28.044 INFO:teuthology.orchestra.run.smithi070.stdout:5411 read 0x8e07b29 thru 0x8e118ec (0x9dc4 bytes) 2024-09-16T11:44:28.044 INFO:teuthology.orchestra.run.smithi070.stdout:5412 write 0x9f4390 thru 0x9faf5f (0x6bd0 bytes) 2024-09-16T11:44:28.046 INFO:teuthology.orchestra.run.smithi070.stdout:5413 read 0xe2427bd thru 0xe248b75 (0x63b9 bytes) 2024-09-16T11:44:28.046 INFO:teuthology.orchestra.run.smithi070.stdout:5414 write 0x234152e thru 0x23492d4 (0x7da7 bytes) 2024-09-16T11:44:28.061 INFO:teuthology.orchestra.run.smithi070.stdout:5415 trunc from 0xed42f6f to 0x32e8a8b 2024-09-16T11:44:28.067 INFO:teuthology.orchestra.run.smithi070.stdout:5416 write 0x48a188 thru 0x491e77 (0x7cf0 bytes) 2024-09-16T11:44:28.069 INFO:teuthology.orchestra.run.smithi070.stdout:5418 punch from 0xc031e8 to 0xc10701, (0xd519 bytes) 2024-09-16T11:44:28.069 INFO:teuthology.orchestra.run.smithi070.stdout:5421 read 0x28653dc thru 0x286a267 (0x4e8c bytes) 2024-09-16T11:44:28.084 INFO:teuthology.orchestra.run.smithi070.stdout:5422 write 0xb5ac1d9 thru 0xb5b6651 (0xa479 bytes) 2024-09-16T11:44:28.087 INFO:teuthology.orchestra.run.smithi070.stdout:5424 write 0x3006a08 thru 0x301422a (0xd823 bytes) 2024-09-16T11:44:28.089 INFO:teuthology.orchestra.run.smithi070.stdout:5425 writesame 0x15e76f6 thru 0x15ee6e8 data_size 0x29(0x6ff3 bytes) 2024-09-16T11:44:28.090 INFO:teuthology.orchestra.run.smithi070.stdout:5426 write 0x91f9201 thru 0x92088cf (0xf6cf bytes) 2024-09-16T11:44:28.117 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:28.114+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:28.136 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:28.136 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:28.136 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:28.136 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:28.144 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:28.218 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:28.217+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:28.281 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:28.281 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:28.281 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:28.281 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:28.281 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:28.289 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:28.319 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:28.317+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:28.387 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:28.387 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16853 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3375104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.037415179, 2024-09-16T11:44:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027791907000000001, 2024-09-16T11:44:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.388 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16853 2024-09-16T11:44:28.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.389 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.389 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.389 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:28.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.389 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.389 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347568+0000", 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347570+0000", 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347588+0000", 2024-09-16T11:44:28.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7914000000000002e-05 2024-09-16T11:44:28.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347592+0000", 2024-09-16T11:44:28.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3150000000000002e-06 2024-09-16T11:44:28.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375199+0000", 2024-09-16T11:44:28.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027606065999999999 2024-09-16T11:44:28.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375276+0000", 2024-09-16T11:44:28.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7527000000000005e-05 2024-09-16T11:44:28.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375359+0000", 2024-09-16T11:44:28.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3238000000000005e-05 2024-09-16T11:44:28.392 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16854 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.037385481999998, 2024-09-16T11:44:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027945148999999999, 2024-09-16T11:44:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16854 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347598+0000", 2024-09-16T11:44:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:44:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347600+0000", 2024-09-16T11:44:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:44:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347606+0000", 2024-09-16T11:44:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0580000000000003e-06 2024-09-16T11:44:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347609+0000", 2024-09-16T11:44:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:44:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375386+0000", 2024-09-16T11:44:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027776809 2024-09-16T11:44:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375461+0000", 2024-09-16T11:44:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4868000000000002e-05 2024-09-16T11:44:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375542+0000", 2024-09-16T11:44:28.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1253000000000003e-05 2024-09-16T11:44:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16855 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 4120576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.037369142999999, 2024-09-16T11:44:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028111094999999999, 2024-09-16T11:44:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.398 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16855 2024-09-16T11:44:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:28.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347614+0000", 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347616+0000", 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347623+0000", 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0099999999999998e-06 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347626+0000", 2024-09-16T11:44:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2140000000000001e-06 2024-09-16T11:44:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375569+0000", 2024-09-16T11:44:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027942649999999999 2024-09-16T11:44:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375643+0000", 2024-09-16T11:44:28.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3949999999999995e-05 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375725+0000", 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1992999999999999e-05 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16856 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3801088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.037347072000003, 2024-09-16T11:44:28.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028268272000000001, 2024-09-16T11:44:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16856 2024-09-16T11:44:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:28.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347636+0000", 2024-09-16T11:44:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:44:28.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347638+0000", 2024-09-16T11:44:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.86e-06 2024-09-16T11:44:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347644+0000", 2024-09-16T11:44:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1739999999999997e-06 2024-09-16T11:44:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347656+0000", 2024-09-16T11:44:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2198000000000001e-05 2024-09-16T11:44:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375751+0000", 2024-09-16T11:44:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028094507000000001 2024-09-16T11:44:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375824+0000", 2024-09-16T11:44:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3178999999999998e-05 2024-09-16T11:44:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375904+0000", 2024-09-16T11:44:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9985000000000001e-05 2024-09-16T11:44:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.408 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16857 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.037320569999999, 2024-09-16T11:44:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028275403000000001, 2024-09-16T11:44:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16857 2024-09-16T11:44:28.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.46e-07 2024-09-16T11:44:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347666+0000", 2024-09-16T11:44:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8739999999999997e-06 2024-09-16T11:44:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347672+0000", 2024-09-16T11:44:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1059999999999996e-06 2024-09-16T11:44:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347676+0000", 2024-09-16T11:44:28.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:44:28.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375930+0000", 2024-09-16T11:44:28.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028254042 2024-09-16T11:44:28.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375937+0000", 2024-09-16T11:44:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5519999999999999e-06 2024-09-16T11:44:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16858 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.037302924999999, 2024-09-16T11:44:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028284592000000001, 2024-09-16T11:44:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:28.416 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16858 2024-09-16T11:44:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:44:28.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347683+0000", 2024-09-16T11:44:28.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6809999999999998e-06 2024-09-16T11:44:28.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347689+0000", 2024-09-16T11:44:28.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347695+0000", 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0020000000000003e-06 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375958+0000", 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028263870999999999 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375964+0000", 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8209999999999998e-06 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16992 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.035728237000001, 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027704776, 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.422 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16992 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349255+0000", 2024-09-16T11:44:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:44:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349260+0000", 2024-09-16T11:44:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5589999999999996e-06 2024-09-16T11:44:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349267+0000", 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4019999999999999e-06 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349270+0000", 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6780000000000002e-06 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376953+0000", 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682933 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376959+0000", 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16993 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.035706912000002, 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027707908999999999, 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.426 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16993 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349278+0000", 2024-09-16T11:44:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.629e-06 2024-09-16T11:44:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349291+0000", 2024-09-16T11:44:28.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614e-05 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349295+0000", 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1189999999999999e-06 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376978+0000", 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682445 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376984+0000", 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7590000000000003e-06 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16994 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.035682901000001, 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027708753999999999, 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.430 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16994 2024-09-16T11:44:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:28.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:44:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349302+0000", 2024-09-16T11:44:28.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.722e-06 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349308+0000", 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349312+0000", 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5310000000000002e-06 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377003+0000", 2024-09-16T11:44:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027691308000000001 2024-09-16T11:44:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377008+0000", 2024-09-16T11:44:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4550000000000003e-06 2024-09-16T11:44:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16999 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.434 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.035540023000003, 2024-09-16T11:44:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027705270000000001, 2024-09-16T11:44:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16999 2024-09-16T11:44:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.46e-07 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349445+0000", 2024-09-16T11:44:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.637e-06 2024-09-16T11:44:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349451+0000", 2024-09-16T11:44:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9959999999999999e-06 2024-09-16T11:44:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349454+0000", 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.647e-06 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377142+0000", 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027687942 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377148+0000", 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7019999999999998e-06 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.438 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17000 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.035524516000002, 2024-09-16T11:44:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.02771881, 2024-09-16T11:44:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:28.439 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17000 2024-09-16T11:44:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:28.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0600000000000001e-07 2024-09-16T11:44:28.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349460+0000", 2024-09-16T11:44:28.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.736e-06 2024-09-16T11:44:28.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349466+0000", 2024-09-16T11:44:28.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2419999999999997e-06 2024-09-16T11:44:28.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349470+0000", 2024-09-16T11:44:28.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5480000000000002e-06 2024-09-16T11:44:28.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377171+0000", 2024-09-16T11:44:28.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027701282000000001 2024-09-16T11:44:28.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377177+0000", 2024-09-16T11:44:28.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5960000000000003e-06 2024-09-16T11:44:28.445 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.445 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.446 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.446 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17001 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.446 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:28.446 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.035508766, 2024-09-16T11:44:28.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027739243, 2024-09-16T11:44:28.447 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.447 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:28.447 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.447 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.447 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.447 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17001 2024-09-16T11:44:28.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.447 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:28.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:28.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:44:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349476+0000", 2024-09-16T11:44:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T11:44:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349482+0000", 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2389999999999999e-06 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349486+0000", 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5980000000000001e-06 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377206+0000", 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027720037999999999 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377213+0000", 2024-09-16T11:44:28.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3440000000000003e-06 2024-09-16T11:44:28.451 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.451 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.451 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.451 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17433 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 5898240~4888] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.451 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:28.451 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.999545112, 2024-09-16T11:44:28.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027709398, 2024-09-16T11:44:28.451 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.451 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.452 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.452 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.452 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.452 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17433 2024-09-16T11:44:28.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.452 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:28.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:28.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385438+0000", 2024-09-16T11:44:28.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:28.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385439+0000", 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5120000000000001e-06 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385445+0000", 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4e-06 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385448+0000", 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1669999999999999e-06 2024-09-16T11:44:28.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412916+0000", 2024-09-16T11:44:28.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027468137 2024-09-16T11:44:28.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412988+0000", 2024-09-16T11:44:28.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1606000000000005e-05 2024-09-16T11:44:28.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413147+0000", 2024-09-16T11:44:28.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015924299999999999 2024-09-16T11:44:28.456 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.456 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.456 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.456 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17434 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6004736~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.456 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:28.456 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.999530522000001, 2024-09-16T11:44:28.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027777461, 2024-09-16T11:44:28.457 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.457 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.457 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.457 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.457 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.457 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17434 2024-09-16T11:44:28.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.457 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:28.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385454+0000", 2024-09-16T11:44:28.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6729999999999999e-06 2024-09-16T11:44:28.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385459+0000", 2024-09-16T11:44:28.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4079999999999997e-06 2024-09-16T11:44:28.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385463+0000", 2024-09-16T11:44:28.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.269e-06 2024-09-16T11:44:28.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413157+0000", 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027694611000000001 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413193+0000", 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5673999999999998e-05 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413230+0000", 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6454e-05 2024-09-16T11:44:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.461 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.461 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.461 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.461 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17435 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6324224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.461 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:28.461 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.999516368999998, 2024-09-16T11:44:28.461 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929059999999999, 2024-09-16T11:44:28.461 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.461 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.461 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.462 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.462 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.462 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17435 2024-09-16T11:44:28.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.462 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.462 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:28.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385467+0000", 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385468+0000", 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6300000000000001e-06 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385474+0000", 2024-09-16T11:44:28.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1299999999999998e-06 2024-09-16T11:44:28.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385478+0000", 2024-09-16T11:44:28.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2880000000000001e-06 2024-09-16T11:44:28.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413257+0000", 2024-09-16T11:44:28.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027779608000000001 2024-09-16T11:44:28.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413331+0000", 2024-09-16T11:44:28.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3639000000000004e-05 2024-09-16T11:44:28.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413395+0000", 2024-09-16T11:44:28.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4424999999999994e-05 2024-09-16T11:44:28.465 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.465 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17436 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7176192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.999500707000003, 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028016718999999999, 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17436 2024-09-16T11:44:28.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.467 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:28.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:28.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385484+0000", 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5179999999999999e-06 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385489+0000", 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7049999999999996e-06 2024-09-16T11:44:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385493+0000", 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413422+0000", 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929611999999999 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413462+0000", 2024-09-16T11:44:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9585000000000001e-05 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413499+0000", 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6644999999999999e-05 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17437 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6750208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.999486292, 2024-09-16T11:44:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028167168999999999, 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17437 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385497+0000", 2024-09-16T11:44:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:44:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385498+0000", 2024-09-16T11:44:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.592e-06 2024-09-16T11:44:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385504+0000", 2024-09-16T11:44:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4090000000000002e-06 2024-09-16T11:44:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385513+0000", 2024-09-16T11:44:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4460000000000007e-06 2024-09-16T11:44:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413526+0000", 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028012866000000001 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413601+0000", 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5388000000000004e-05 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413663+0000", 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.211e-05 2024-09-16T11:44:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17438 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7495680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.999463532999997, 2024-09-16T11:44:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028224439, 2024-09-16T11:44:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17438 2024-09-16T11:44:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4900000000000001e-07 2024-09-16T11:44:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385521+0000", 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385526+0000", 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2680000000000002e-06 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385529+0000", 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2040000000000002e-06 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413674+0000", 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028144226000000001 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413710+0000", 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6665000000000002e-05 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413743+0000", 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3155999999999999e-05 2024-09-16T11:44:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17439 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7921664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.999402379000003, 2024-09-16T11:44:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028326131000000001, 2024-09-16T11:44:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17439 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385581+0000", 2024-09-16T11:44:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9299999999999998e-07 2024-09-16T11:44:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385583+0000", 2024-09-16T11:44:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6220000000000001e-06 2024-09-16T11:44:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385592+0000", 2024-09-16T11:44:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5220000000000007e-06 2024-09-16T11:44:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385595+0000", 2024-09-16T11:44:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8669999999999998e-06 2024-09-16T11:44:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413771+0000", 2024-09-16T11:44:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028175875999999999 2024-09-16T11:44:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413845+0000", 2024-09-16T11:44:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4382999999999996e-05 2024-09-16T11:44:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413906+0000", 2024-09-16T11:44:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1367999999999999e-05 2024-09-16T11:44:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17440 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.999385171, 2024-09-16T11:44:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028391117, 2024-09-16T11:44:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17440 2024-09-16T11:44:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.485 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385598+0000", 2024-09-16T11:44:28.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385603+0000", 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385607+0000", 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4869999999999998e-06 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385609+0000", 2024-09-16T11:44:28.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2759999999999999e-06 2024-09-16T11:44:28.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413916+0000", 2024-09-16T11:44:28.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028307253000000001 2024-09-16T11:44:28.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413955+0000", 2024-09-16T11:44:28.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8652000000000002e-05 2024-09-16T11:44:28.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413989+0000", 2024-09-16T11:44:28.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3587999999999998e-05 2024-09-16T11:44:28.489 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.489 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.489 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.489 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.489 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.489 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:28.491 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:28.491 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:28.488+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:28.591 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:28.589+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:28.629 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55904 4.14 4.1c7fb4b4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.53679824200000004, 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.53681697299999998, 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for peered", 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55904 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091737+0000", 2024-09-16T11:44:28.631 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.254e-06 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091759+0000", 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2487999999999999e-05 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091769+0000", 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0199e-05 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091774+0000", 2024-09-16T11:44:28.632 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091830+0000", 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6700999999999998e-05 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091840+0000", 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0128e-05 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.633 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.634 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:28.634 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 1 2024-09-16T11:44:28.634 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:28.638 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:28.692 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:28.691+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:28.775 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:28.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:28.827 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:28.825+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:28.882 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19545 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4415488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.355771636, 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043429581000000002, 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19545 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523839+0000", 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0220000000000004e-06 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523845+0000", 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6500000000000001e-06 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523848+0000", 2024-09-16T11:44:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1879999999999998e-06 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567092+0000", 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043244145999999997 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567176+0000", 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3825999999999993e-05 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567263+0000", 2024-09-16T11:44:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7423999999999995e-05 2024-09-16T11:44:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19546 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4734976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.355753796000002, 2024-09-16T11:44:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043599231000000002, 2024-09-16T11:44:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19546 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523954+0000", 2024-09-16T11:44:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010217099999999999 2024-09-16T11:44:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523957+0000", 2024-09-16T11:44:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3270000000000001e-06 2024-09-16T11:44:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523966+0000", 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6489999999999997e-06 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523971+0000", 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567291+0000", 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043319931999999998 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567360+0000", 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8628999999999996e-05 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567451+0000", 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1012000000000002e-05 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19547 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4841472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.355628938999999, 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043658858000000002, 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19547 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525102+0000", 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011257730000000001 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525106+0000", 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9910000000000004e-06 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525116+0000", 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0231000000000001e-05 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525137+0000", 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0828000000000001e-05 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567478+0000", 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.042340467999999999 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567554+0000", 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6074000000000004e-05 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567635+0000", 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1493000000000001e-05 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19553 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4947968~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.354401877000001, 2024-09-16T11:44:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043450397000000002, 2024-09-16T11:44:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19553 2024-09-16T11:44:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1799999999999999e-07 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525205+0000", 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.062e-06 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525208+0000", 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8739999999999999e-06 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525209+0000", 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.725e-06 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568520+0000", 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043310226 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568569+0000", 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8959999999999999e-05 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568654+0000", 2024-09-16T11:44:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5332000000000002e-05 2024-09-16T11:44:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19554 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5267456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.354392885000003, 2024-09-16T11:44:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043659587, 2024-09-16T11:44:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19554 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527103+0000", 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018907419999999999 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527108+0000", 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6040000000000003e-06 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527118+0000", 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527122+0000", 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9559999999999996e-06 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568688+0000", 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041565116999999999 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568768+0000", 2024-09-16T11:44:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0474000000000002e-05 2024-09-16T11:44:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568872+0000", 2024-09-16T11:44:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104028 2024-09-16T11:44:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19565 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6332416~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.352314313000001, 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043494890000000001, 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19565 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527292+0000", 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.27e-07 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527293+0000", 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.917e-06 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527299+0000", 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6699999999999999e-06 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527304+0000", 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6160000000000004e-06 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570683+0000", 2024-09-16T11:44:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043379171000000001 2024-09-16T11:44:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570733+0000", 2024-09-16T11:44:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0485000000000002e-05 2024-09-16T11:44:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570786+0000", 2024-09-16T11:44:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2604e-05 2024-09-16T11:44:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19566 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6758400~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.352297978999999, 2024-09-16T11:44:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043643183000000002, 2024-09-16T11:44:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19566 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527308+0000", 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527310+0000", 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.68e-06 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527316+0000", 2024-09-16T11:44:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9730000000000002e-06 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527319+0000", 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.027e-06 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570810+0000", 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043491525000000003 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570882+0000", 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570951+0000", 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8782000000000007e-05 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19567 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.352283272999998, 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043786291999999997, 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19567 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529108+0000", 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001785656 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529113+0000", 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1080000000000001e-06 2024-09-16T11:44:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529122+0000", 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0550000000000005e-06 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529127+0000", 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9849999999999997e-06 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570997+0000", 2024-09-16T11:44:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041869811 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571063+0000", 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6340999999999994e-05 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571108+0000", 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5336000000000001e-05 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19580 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 8036352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.350307972000003, 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043451596000000002, 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19580 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531106+0000", 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001808179 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531110+0000", 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5990000000000004e-06 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531123+0000", 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2687000000000001e-05 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531129+0000", 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:44:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572642+0000", 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041513474000000002 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572691+0000", 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8989000000000001e-05 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572749+0000", 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7806999999999997e-05 2024-09-16T11:44:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.922 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.922 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.922 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19708 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4530176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.922 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:28.922 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.348261407000003, 2024-09-16T11:44:28.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394863999999998, 2024-09-16T11:44:28.922 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.922 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.922 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.922 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.922 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19708 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531344+0000", 2024-09-16T11:44:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.67e-07 2024-09-16T11:44:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531346+0000", 2024-09-16T11:44:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.477e-06 2024-09-16T11:44:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531351+0000", 2024-09-16T11:44:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1050000000000003e-06 2024-09-16T11:44:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531354+0000", 2024-09-16T11:44:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9960000000000002e-06 2024-09-16T11:44:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574620+0000", 2024-09-16T11:44:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043266365000000001 2024-09-16T11:44:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574662+0000", 2024-09-16T11:44:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1662000000000002e-05 2024-09-16T11:44:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574739+0000", 2024-09-16T11:44:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6991999999999997e-05 2024-09-16T11:44:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19709 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4849664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.348247931000003, 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043537291999999998, 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19709 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531358+0000", 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531360+0000", 2024-09-16T11:44:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:44:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531365+0000", 2024-09-16T11:44:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.716e-06 2024-09-16T11:44:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531369+0000", 2024-09-16T11:44:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5410000000000001e-06 2024-09-16T11:44:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574763+0000", 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394359 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574831+0000", 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7997999999999995e-05 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574895+0000", 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3427000000000001e-05 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19710 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4956160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.348232652, 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043674198999999997, 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19710 2024-09-16T11:44:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533105+0000", 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001731961 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533109+0000", 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3080000000000001e-06 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533120+0000", 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0625000000000001e-05 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533125+0000", 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574923+0000", 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041797187 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574991+0000", 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8752999999999997e-05 2024-09-16T11:44:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.575047+0000", 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5596000000000001e-05 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19719 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5488640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.346260903000001, 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043412013999999999, 2024-09-16T11:44:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19719 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533345+0000", 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533347+0000", 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8139999999999999e-06 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533352+0000", 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4970000000000001e-06 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533355+0000", 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.711e-06 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576629+0000", 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043274295999999997 2024-09-16T11:44:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576706+0000", 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6711000000000005e-05 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576757+0000", 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0509e-05 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19720 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5808128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.346248254000002, 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043577006000000001, 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19720 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.35e-07 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533359+0000", 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533364+0000", 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4349999999999996e-06 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533367+0000", 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2349999999999999e-06 2024-09-16T11:44:28.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576786+0000", 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043418885999999997 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576862+0000", 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6172000000000002e-05 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576934+0000", 2024-09-16T11:44:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1891999999999995e-05 2024-09-16T11:44:28.944 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.944 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.944 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.944 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19721 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6234112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.944 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:28.944 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.346234733999999, 2024-09-16T11:44:28.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043681656999999999, 2024-09-16T11:44:28.944 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.944 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.944 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.944 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19721 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-07 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533373+0000", 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.973e-06 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533377+0000", 2024-09-16T11:44:28.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.403e-06 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533379+0000", 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.058e-06 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576959+0000", 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043579433000000001 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577015+0000", 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5859000000000002e-05 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577052+0000", 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7620999999999998e-05 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19722 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6340608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:28.948 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.346224007000004, 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043854922999999997, 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19722 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:28.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535103+0000", 2024-09-16T11:44:28.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017217020000000001 2024-09-16T11:44:28.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535107+0000", 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2819999999999998e-06 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535118+0000", 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0148e-05 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535123+0000", 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.952 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577091+0000", 2024-09-16T11:44:28.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041967930000000001 2024-09-16T11:44:28.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.952 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.952 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577163+0000", 2024-09-16T11:44:28.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1853999999999993e-05 2024-09-16T11:44:28.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.952 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.952 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577236+0000", 2024-09-16T11:44:28.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3699e-05 2024-09-16T11:44:28.952 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19732 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6447104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.344364151000001, 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043399048000000003, 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.953 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19732 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535242+0000", 2024-09-16T11:44:28.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:28.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535243+0000", 2024-09-16T11:44:28.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.793e-06 2024-09-16T11:44:28.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535249+0000", 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.643e-06 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535252+0000", 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578501+0000", 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043248343000000002 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.957 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.957 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578576+0000", 2024-09-16T11:44:28.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5544999999999996e-05 2024-09-16T11:44:28.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.957 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.957 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578640+0000", 2024-09-16T11:44:28.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4018999999999998e-05 2024-09-16T11:44:28.957 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.957 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.957 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19733 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6873088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.344343309999999, 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043549681, 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19733 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535263+0000", 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535265+0000", 2024-09-16T11:44:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9369999999999998e-06 2024-09-16T11:44:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535271+0000", 2024-09-16T11:44:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9719999999999997e-06 2024-09-16T11:44:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535274+0000", 2024-09-16T11:44:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:44:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578668+0000", 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043393355000000002 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578744+0000", 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6853000000000004e-05 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578812+0000", 2024-09-16T11:44:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.737e-05 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19734 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7299072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.344327563, 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043699189999999999, 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.962 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19734 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537101+0000", 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001823098 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537105+0000", 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537119+0000", 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614999999999999e-05 2024-09-16T11:44:28.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537125+0000", 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578838+0000", 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041712605999999999 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578912+0000", 2024-09-16T11:44:28.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4396999999999999e-05 2024-09-16T11:44:28.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.966 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.966 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578977+0000", 2024-09-16T11:44:28.966 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4858999999999997e-05 2024-09-16T11:44:28.966 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.966 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.966 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.966 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19851 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4431872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:28.966 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:28.966 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 33.342246819000003, 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043564540999999998, 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19851 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:28.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:28.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:28.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:28.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539100+0000", 2024-09-16T11:44:28.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017412079999999999 2024-09-16T11:44:28.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:28.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539105+0000", 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539116+0000", 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1824999999999999e-05 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539122+0000", 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9859999999999999e-06 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580764+0000", 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041641299 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580839+0000", 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5167999999999996e-05 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580923+0000", 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4314999999999995e-05 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.970 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.971 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:28.971 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:28.971 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:28.972 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:28.972 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:28.970+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:28.993 INFO:teuthology.orchestra.run.smithi070.stdout:5427 read 0x3d31077 thru 0x3d320f0 (0x107a bytes) 2024-09-16T11:44:28.995 INFO:teuthology.orchestra.run.smithi070.stdout:5429 read 0x6cd21e2 thru 0x6cd4445 (0x2264 bytes) 2024-09-16T11:44:28.995 INFO:teuthology.orchestra.run.smithi070.stdout:5430 write 0x2023ffc thru 0x202d651 (0x9656 bytes) 2024-09-16T11:44:28.999 INFO:teuthology.orchestra.run.smithi070.stdout:5431 write 0x4caaa77 thru 0x4cb45fb (0x9b85 bytes) 2024-09-16T11:44:29.002 INFO:teuthology.orchestra.run.smithi070.stdout:5432 write 0x9674bd5 thru 0x967b7cd (0x6bf9 bytes) 2024-09-16T11:44:29.005 INFO:teuthology.orchestra.run.smithi070.stdout:5434 writesame 0xea3d4d thru 0xeaf5aa data_size 0x1(0xb85e bytes) 2024-09-16T11:44:29.029 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.029 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.029 INFO:teuthology.orchestra.run.smithi070.stdout:5438 punch from 0x88fe2ea to 0x8907742, (0x9458 bytes) 2024-09-16T11:44:29.029 INFO:teuthology.orchestra.run.smithi070.stdout:5440 read 0x5b1a2bf thru 0x5b26f1c (0xcc5e bytes) 2024-09-16T11:44:29.029 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.030 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.030 INFO:teuthology.orchestra.run.smithi070.stdout:5443 read 0xa5bb73a thru 0xa5bc21a (0xae1 bytes) 2024-09-16T11:44:29.032 INFO:teuthology.orchestra.run.smithi070.stdout:5444 trunc from 0xb5b6652 to 0xe36ecf0 2024-09-16T11:44:29.038 INFO:teuthology.orchestra.run.smithi070.stdout:5445 write 0xdf6014a thru 0xdf6ed77 (0xec2e bytes) 2024-09-16T11:44:29.041 INFO:teuthology.orchestra.run.smithi070.stdout:5447 write 0x187b370 thru 0x187e69c (0x332d bytes) 2024-09-16T11:44:29.042 INFO:teuthology.orchestra.run.smithi070.stdout:5448 write 0xb2b4bfe thru 0xb2bf721 (0xab24 bytes) 2024-09-16T11:44:29.045 INFO:teuthology.orchestra.run.smithi070.stdout:5449 write 0x66a462e thru 0x66aabcf (0x65a2 bytes) 2024-09-16T11:44:29.048 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.048 INFO:teuthology.orchestra.run.smithi070.stdout:5451 write 0x4fac50 thru 0x50213f (0x74f0 bytes) 2024-09-16T11:44:29.050 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.050 INFO:teuthology.orchestra.run.smithi070.stdout:5455 read 0xd88b598 thru 0xd88fbd7 (0x4640 bytes) 2024-09-16T11:44:29.050 INFO:teuthology.orchestra.run.smithi070.stdout:5456 read 0x969f4ea thru 0x969f4fa (0x11 bytes) 2024-09-16T11:44:29.050 INFO:teuthology.orchestra.run.smithi070.stdout:5458 punch from 0xc92d1fd to 0xc939a82, (0xc885 bytes) 2024-09-16T11:44:29.050 INFO:teuthology.orchestra.run.smithi070.stdout:5459 write 0xc9c1ee1 thru 0xc9c469d (0x27bd bytes) 2024-09-16T11:44:29.063 INFO:teuthology.orchestra.run.smithi070.stdout:5461 trunc from 0xe36ecf0 to 0x5fbb18d 2024-09-16T11:44:29.074 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:29.072+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:29.079 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:29.079 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:29.079 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:29.082 INFO:teuthology.orchestra.run.smithi070.stdout:5463 write 0xae1bf39 thru 0xae2045f (0x4527 bytes) 2024-09-16T11:44:29.085 INFO:teuthology.orchestra.run.smithi070.stdout:5464 write 0x3e764a2 thru 0x3e77298 (0xdf7 bytes) 2024-09-16T11:44:29.088 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:29.097 INFO:teuthology.orchestra.run.smithi070.stdout:5465 trunc from 0xae20460 to 0x3359cff 2024-09-16T11:44:29.112 INFO:teuthology.orchestra.run.smithi070.stdout:5466 writesame 0x4111826 thru 0x4116dde data_size 0x4d(0x55b9 bytes) 2024-09-16T11:44:29.121 INFO:teuthology.orchestra.run.smithi070.stdout:5467 write 0x5b96b5a thru 0x5b9a14d (0x35f4 bytes) 2024-09-16T11:44:29.138 INFO:teuthology.orchestra.run.smithi070.stdout:5472 writesame 0xdb2a8bf thru 0xdb2c06c data_size 0x7(0x17ae bytes) 2024-09-16T11:44:29.138 INFO:teuthology.orchestra.run.smithi070.stdout:5473 punch from 0x539a2db to 0x53a4751, (0xa476 bytes) 2024-09-16T11:44:29.138 INFO:teuthology.orchestra.run.smithi070.stdout:5474 writesame 0xd3a91ae thru 0xd3b339b data_size 0x15(0xa1ee bytes) 2024-09-16T11:44:29.141 INFO:teuthology.orchestra.run.smithi070.stdout:5475 writesame 0x788fb16 thru 0x789d831 data_size 0x59(0xdd1c bytes) 2024-09-16T11:44:29.152 INFO:teuthology.orchestra.run.smithi070.stdout:5476 trunc from 0xdb2c06d to 0x6e4f817 2024-09-16T11:44:29.160 INFO:teuthology.orchestra.run.smithi070.stdout:5477 read 0x50cbb04 thru 0x50cc48d (0x98a bytes) 2024-09-16T11:44:29.161 INFO:teuthology.orchestra.run.smithi070.stdout:5478 write 0x6b06731 thru 0x6b11c39 (0xb509 bytes) 2024-09-16T11:44:29.174 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:29.172+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:29.175 INFO:teuthology.orchestra.run.smithi070.stdout:5479 write 0xb77854a thru 0xb78340b (0xaec2 bytes) 2024-09-16T11:44:29.177 INFO:teuthology.orchestra.run.smithi070.stdout:5480 read 0x4182454 thru 0x418998c (0x7539 bytes) 2024-09-16T11:44:29.177 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.178 INFO:teuthology.orchestra.run.smithi070.stdout:5482 read 0x6a84c3 thru 0x6ad720 (0x525e bytes) 2024-09-16T11:44:29.178 INFO:teuthology.orchestra.run.smithi070.stdout:5483 writesame 0x897bb79 thru 0x8989046 data_size 0x1(0xd4ce bytes) 2024-09-16T11:44:29.212 INFO:teuthology.orchestra.run.smithi070.stdout:5484 write 0x3788a99 thru 0x378d3d0 (0x4938 bytes) 2024-09-16T11:44:29.216 INFO:teuthology.orchestra.run.smithi070.stdout:5487 trunc from 0xb78340c to 0xbda94e0 2024-09-16T11:44:29.220 INFO:teuthology.orchestra.run.smithi070.stdout:5488 writesame 0x943f122 thru 0x9441842 data_size 0x35(0x2721 bytes) 2024-09-16T11:44:29.223 INFO:teuthology.orchestra.run.smithi070.stdout:5489 read 0xa8d7e00 thru 0xa8e405d (0xc25e bytes) 2024-09-16T11:44:29.223 INFO:teuthology.orchestra.run.smithi070.stdout:5490 read 0x2f36fc6 thru 0x2f3fe54 (0x8e8f bytes) 2024-09-16T11:44:29.223 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.228 INFO:teuthology.orchestra.run.smithi070.stdout:5492 write 0xc41a496 thru 0xc41e9d4 (0x453f bytes) 2024-09-16T11:44:29.231 INFO:teuthology.orchestra.run.smithi070.stdout:5494 writesame 0x2071fdd thru 0x207fcfe data_size 0x33(0xdd22 bytes) 2024-09-16T11:44:29.238 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:29.238 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:29.239 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:29.239 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:29.239 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:29.239 INFO:teuthology.orchestra.run.smithi070.stdout:5497 trunc from 0xc41e9d5 to 0x68e2435 2024-09-16T11:44:29.248 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:29.251 INFO:teuthology.orchestra.run.smithi070.stdout:5498 read 0x40a87b6 thru 0x40aea38 (0x6283 bytes) 2024-09-16T11:44:29.251 INFO:teuthology.orchestra.run.smithi070.stdout:5499 write 0x1999af5 thru 0x19a2f1f (0x942b bytes) 2024-09-16T11:44:29.253 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.253 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.253 INFO:teuthology.orchestra.run.smithi070.stdout:5507 read 0x6006e3d thru 0x600bf96 (0x515a bytes) 2024-09-16T11:44:29.253 INFO:teuthology.orchestra.run.smithi070.stdout:5509 read 0xe8900d thru 0xe8b1d5 (0x21c9 bytes) 2024-09-16T11:44:29.253 INFO:teuthology.orchestra.run.smithi070.stdout:5510 write 0x1c2e5d3 thru 0x1c346c6 (0x60f4 bytes) 2024-09-16T11:44:29.256 INFO:teuthology.orchestra.run.smithi070.stdout:5512 punch from 0x6ba540 to 0x6bcc6f, (0x272f bytes) 2024-09-16T11:44:29.256 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.265 INFO:teuthology.orchestra.run.smithi070.stdout:5514 writesame 0xab5c845 thru 0xab6c5be data_size 0x69(0xfd7a bytes) 2024-09-16T11:44:29.277 INFO:teuthology.orchestra.run.smithi070.stdout:5515 trunc from 0xab6c5bf to 0xee0be5 2024-09-16T11:44:29.289 INFO:teuthology.orchestra.run.smithi070.stdout:5516 trunc from 0xee0be5 to 0x1f13fc4 2024-09-16T11:44:29.291 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:29.290+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:29.300 INFO:teuthology.orchestra.run.smithi070.stdout:5517 trunc from 0x1f13fc4 to 0x7bf3b9f 2024-09-16T11:44:29.306 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.306 INFO:teuthology.orchestra.run.smithi070.stdout:5520 read 0x1710292 thru 0x171dbeb (0xd95a bytes) 2024-09-16T11:44:29.306 INFO:teuthology.orchestra.run.smithi070.stdout:5521 punch from 0x1c376aa to 0x1c4428e, (0xcbe4 bytes) 2024-09-16T11:44:29.306 INFO:teuthology.orchestra.run.smithi070.stdout:5522 read 0x5ae3472 thru 0x5ae8511 (0x50a0 bytes) 2024-09-16T11:44:29.311 INFO:teuthology.orchestra.run.smithi070.stdout:5523 write 0x8c450bf thru 0x8c488d5 (0x3817 bytes) 2024-09-16T11:44:29.323 INFO:teuthology.orchestra.run.smithi070.stdout:5524 write 0xb5ca70c thru 0xb5cf401 (0x4cf6 bytes) 2024-09-16T11:44:29.325 INFO:teuthology.orchestra.run.smithi070.stdout:5525 read 0x31b715e thru 0x31c2583 (0xb426 bytes) 2024-09-16T11:44:29.325 INFO:teuthology.orchestra.run.smithi070.stdout:5526 punch from 0x76a9472 to 0x76b5f0f, (0xca9d bytes) 2024-09-16T11:44:29.325 INFO:teuthology.orchestra.run.smithi070.stdout:5527 read 0x31fac4 thru 0x32c426 (0xc963 bytes) 2024-09-16T11:44:29.326 INFO:teuthology.orchestra.run.smithi070.stdout:5528 write 0xa56af6e thru 0xa56b451 (0x4e4 bytes) 2024-09-16T11:44:29.328 INFO:teuthology.orchestra.run.smithi070.stdout:5530 write 0x125091c thru 0x12540fa (0x37df bytes) 2024-09-16T11:44:29.330 INFO:teuthology.orchestra.run.smithi070.stdout:5531 punch from 0x7fe298f to 0x7fe4e12, (0x2483 bytes) 2024-09-16T11:44:29.339 INFO:teuthology.orchestra.run.smithi070.stdout:5532 trunc from 0xb5cf402 to 0x35fe7c7 2024-09-16T11:44:29.351 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:29.351 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:29.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:29.351 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:29.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.351 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.351 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:29.351 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987302575999998, 2024-09-16T11:44:29.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:44:29.352 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.352 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.352 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:44:29.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.352 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:29.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:44:29.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:44:29.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:44:29.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:44:29.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:44:29.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:44:29.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:44:29.355 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.356 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.356 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.356 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:29.356 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987280546999997, 2024-09-16T11:44:29.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:44:29.356 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.356 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:44:29.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:44:29.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987265782000001, 2024-09-16T11:44:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.364 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.365 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:29.365 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987245713999997, 2024-09-16T11:44:29.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:44:29.365 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.365 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.365 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:44:29.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.365 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:44:29.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:44:29.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:44:29.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:44:29.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:44:29.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:44:29.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:44:29.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987214878000003, 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:44:29.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:44:29.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:44:29.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:44:29.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:44:29.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:29.373 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987195358000001, 2024-09-16T11:44:29.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:44:29.374 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.374 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.374 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:44:29.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.374 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:29.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.377 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.378 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.378 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.378 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:29.378 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987167779000004, 2024-09-16T11:44:29.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:44:29.378 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.378 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.379 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.379 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:44:29.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.379 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:29.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:29.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.382 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987149084000002, 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:29.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:29.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:44:29.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:44:29.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:44:29.386 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.387 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.387 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.387 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.387 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:29.387 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987134075, 2024-09-16T11:44:29.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:44:29.387 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.387 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.388 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:44:29.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.388 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:29.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:29.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:44:29.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:29.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.394 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.395 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.395 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:29.395 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987101977000002, 2024-09-16T11:44:29.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:44:29.395 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.395 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:44:29.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:29.399 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987078357999998, 2024-09-16T11:44:29.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:44:29.400 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.400 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.400 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:44:29.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.400 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:29.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:29.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:29.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:29.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:44:29.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:44:29.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:44:29.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:44:29.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:44:29.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:44:29.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:44:29.403 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.404 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.404 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.404 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.404 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:29.404 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987063628999998, 2024-09-16T11:44:29.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:44:29.404 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.404 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.405 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:44:29.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.405 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:29.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:29.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:44:29.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:44:29.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:44:29.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:44:29.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:44:29.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.408 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.408 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.408 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987041804999997, 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:29.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:29.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:29.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:29.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:44:29.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:44:29.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:44:29.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:44:29.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:44:29.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987027155, 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:44:29.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:44:29.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:44:29.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:44:29.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:44:29.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:44:29.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:44:29.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:44:29.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:44:29.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:44:29.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.419 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.419 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.419 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:29.419 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.987005582000002, 2024-09-16T11:44:29.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:44:29.419 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.419 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.420 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:44:29.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.420 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:29.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:29.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:44:29.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:44:29.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:44:29.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:44:29.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:44:29.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.423 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.423 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.423 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:29.423 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.986986135999999, 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:29.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:44:29.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:29.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:44:29.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:44:29.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:44:29.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:44:29.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:44:29.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:44:29.427 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.427 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.986949439, 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:44:29.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:44:29.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:44:29.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:44:29.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:44:29.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:44:29.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:44:29.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.986927493000003, 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.432 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.433 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:44:29.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.433 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:29.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:44:29.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:44:29.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:44:29.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:44:29.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:44:29.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:44:29.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.437 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.437 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:29.437 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.986905606999997, 2024-09-16T11:44:29.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:44:29.437 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.437 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.437 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:44:29.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.437 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:44:29.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:44:29.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:44:29.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:44:29.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.441 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.441 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:29.441 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:29.441 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.986890887999998, 2024-09-16T11:44:29.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:44:29.441 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:29.441 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:29.441 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:29.441 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:29.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:29.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:29.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:29.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:29.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:44:29.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:44:29.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:29.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:29.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:44:29.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:44:29.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:29.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:29.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:29.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:44:29.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:44:29.445 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.445 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.445 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.445 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:29.445 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:29.445 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:29.448 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:29.449 INFO:teuthology.orchestra.run.smithi070.stdout:5533 writesame 0x81bb3b6 thru 0x81c7aac data_size 0x3d(0xc6f7 bytes) 2024-09-16T11:44:29.449 INFO:teuthology.orchestra.run.smithi070.stdout:5536 trunc from 0x81c7aad to 0x13b05d6 2024-09-16T11:44:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:5537 write 0x44fa1ec thru 0x4501589 (0x739e bytes) 2024-09-16T11:44:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:5538 write 0xb4377c5 thru 0xb446dee (0xf62a bytes) 2024-09-16T11:44:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:5539 write 0xe8694fa thru 0xe86f8cd (0x63d4 bytes) 2024-09-16T11:44:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:5541 write 0x639fcb7 thru 0x63a21d8 (0x2522 bytes) 2024-09-16T11:44:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:5546 trunc from 0xe86f8ce to 0xe2fecd4 2024-09-16T11:44:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:5547 punch from 0xc8e4387 to 0xc8f14df, (0xd158 bytes) 2024-09-16T11:44:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:5549 write 0xdeef2f7 thru 0xdef05d0 (0x12da bytes) 2024-09-16T11:44:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:5550 punch from 0x2627c12 to 0x262ca2c, (0x4e1a bytes) 2024-09-16T11:44:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:5551 punch from 0x30a53f6 to 0x30b1084, (0xbc8e bytes) 2024-09-16T11:44:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:5552 read 0xad85bb2 thru 0xad86b95 (0xfe4 bytes) 2024-09-16T11:44:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:5553 write 0xd806954 thru 0xd80a627 (0x3cd4 bytes) 2024-09-16T11:44:29.451 INFO:teuthology.orchestra.run.smithi070.stdout:5554 write 0xebbe1ce thru 0xebbf144 (0xf77 bytes) 2024-09-16T11:44:29.451 INFO:teuthology.orchestra.run.smithi070.stdout:5556 punch from 0x3b6e943 to 0x3b7a75c, (0xbe19 bytes) 2024-09-16T11:44:29.451 INFO:teuthology.orchestra.run.smithi070.stdout:5558 writesame 0x66233aa thru 0x66290c3 data_size 0x11(0x5d1a bytes) 2024-09-16T11:44:29.451 INFO:teuthology.orchestra.run.smithi070.stdout:5559 punch from 0x454d63d to 0x455b797, (0xe15a bytes) 2024-09-16T11:44:29.451 INFO:teuthology.orchestra.run.smithi070.stdout:5560 read 0xb08218d thru 0xb086b4b (0x49bf bytes) 2024-09-16T11:44:29.451 INFO:teuthology.orchestra.run.smithi070.stdout:5561 punch from 0x1526a49 to 0x152c9f0, (0x5fa7 bytes) 2024-09-16T11:44:29.451 INFO:teuthology.orchestra.run.smithi070.stdout:5562 write 0x4b286de thru 0x4b3473b (0xc05e bytes) 2024-09-16T11:44:29.451 INFO:teuthology.orchestra.run.smithi070.stdout:5565 read 0x9faa760 thru 0x9fb85b8 (0xde59 bytes) 2024-09-16T11:44:29.451 INFO:teuthology.orchestra.run.smithi070.stdout:5566 read 0x389776e thru 0x389c38e (0x4c21 bytes) 2024-09-16T11:44:29.451 INFO:teuthology.orchestra.run.smithi070.stdout:5567 trunc from 0xebbf145 to 0x9fb32c9 2024-09-16T11:44:29.451 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:29.444+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:29.454 INFO:teuthology.orchestra.run.smithi070.stdout:5568 read 0x718838e thru 0x7189553 (0x11c6 bytes) 2024-09-16T11:44:29.454 INFO:teuthology.orchestra.run.smithi070.stdout:5569 write 0x6041175 thru 0x604e449 (0xd2d5 bytes) 2024-09-16T11:44:29.457 INFO:teuthology.orchestra.run.smithi070.stdout:5574 write 0x7acecdc thru 0x7ad70dd (0x8402 bytes) 2024-09-16T11:44:29.460 INFO:teuthology.orchestra.run.smithi070.stdout:5575 read 0x3eb3d8e thru 0x3eb4ed4 (0x1147 bytes) 2024-09-16T11:44:29.460 INFO:teuthology.orchestra.run.smithi070.stdout:5578 writesame 0x2d9eab4 thru 0x2da8b08 data_size 0x1(0xa055 bytes) 2024-09-16T11:44:29.481 INFO:teuthology.orchestra.run.smithi070.stdout:5579 read 0x45d67c thru 0x46034c (0x2cd1 bytes) 2024-09-16T11:44:29.481 INFO:teuthology.orchestra.run.smithi070.stdout:5582 writesame 0x24af72c thru 0x24bb706 data_size 0x37(0xbfdb bytes) 2024-09-16T11:44:29.485 INFO:teuthology.orchestra.run.smithi070.stdout:5584 writesame 0x971d8a6 thru 0x97276c4 data_size 0x67(0x9e1f bytes) 2024-09-16T11:44:29.488 INFO:teuthology.orchestra.run.smithi070.stdout:5585 write 0x7d79342 thru 0x7d808ac (0x756b bytes) 2024-09-16T11:44:29.491 INFO:teuthology.orchestra.run.smithi070.stdout:5586 punch from 0x80528c7 to 0x805cd65, (0xa49e bytes) 2024-09-16T11:44:29.491 INFO:teuthology.orchestra.run.smithi070.stdout:5588 write 0x3b8b852 thru 0x3b965d7 (0xad86 bytes) 2024-09-16T11:44:29.497 INFO:teuthology.orchestra.run.smithi070.stdout:5590 trunc from 0x9fb32c9 to 0xc7aad17 2024-09-16T11:44:29.507 INFO:teuthology.orchestra.run.smithi070.stdout:5592 writesame 0xcefab50 thru 0xcefacfd data_size 0x5(0x1ae bytes) 2024-09-16T11:44:29.507 INFO:teuthology.orchestra.run.smithi070.stdout:5593 write 0x7177a27 thru 0x717b786 (0x3d60 bytes) 2024-09-16T11:44:29.510 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.510 INFO:teuthology.orchestra.run.smithi070.stdout:5596 writesame 0x6bf0cb6 thru 0x6bff681 data_size 0x1a(0xe9cc bytes) 2024-09-16T11:44:29.513 INFO:teuthology.orchestra.run.smithi070.stdout:5597 read 0x28e2691 thru 0x28edd8a (0xb6fa bytes) 2024-09-16T11:44:29.513 INFO:teuthology.orchestra.run.smithi070.stdout:5598 write 0x5ea63c9 thru 0x5eaa9a7 (0x45df bytes) 2024-09-16T11:44:29.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:44:29.516 INFO:teuthology.orchestra.run.smithi070.stdout:5599 read 0x56078cb thru 0x5608707 (0xe3d bytes) 2024-09-16T11:44:29.528 INFO:teuthology.orchestra.run.smithi070.stdout:5600 trunc from 0xcefacfe to 0x2a5b50b 2024-09-16T11:44:29.548 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:29.545+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:29.552 INFO:teuthology.orchestra.run.smithi070.stdout:5601 writesame 0x87d1974 thru 0x87dfd12 data_size 0x1(0xe39f bytes) 2024-09-16T11:44:29.557 INFO:teuthology.orchestra.run.smithi070.stdout:5603 write 0x8f8b8bc thru 0x8f9996a (0xe0af bytes) 2024-09-16T11:44:29.560 INFO:teuthology.orchestra.run.smithi070.stdout:5605 writesame 0x6d2aec1 thru 0x6d2d04c data_size 0x4(0x218c bytes) 2024-09-16T11:44:29.563 INFO:teuthology.orchestra.run.smithi070.stdout:5606 writesame 0x2b3d858 thru 0x2b455d9 data_size 0x99(0x7d82 bytes) 2024-09-16T11:44:29.566 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.566 INFO:teuthology.orchestra.run.smithi070.stdout:5609 flatten 2024-09-16T11:44:29.571 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.572 INFO:teuthology.orchestra.run.smithi070.stdout:5612 trunc from 0x8f9996b to 0x7f4d250 2024-09-16T11:44:29.581 INFO:teuthology.orchestra.run.smithi070.stdout:5613 punch from 0x3f4c92f to 0x3f501e3, (0x38b4 bytes) 2024-09-16T11:44:29.590 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:29.590 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:29.590 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:29.590 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:29.591 INFO:teuthology.orchestra.run.smithi070.stdout:5615 trunc from 0x7f4d250 to 0x7526b7 2024-09-16T11:44:29.599 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:29.605 INFO:teuthology.orchestra.run.smithi070.stdout:5616 punch from 0x12aca4 to 0x12f7a9, (0x4b05 bytes) 2024-09-16T11:44:29.605 INFO:teuthology.orchestra.run.smithi070.stdout:5617 read 0x24107e thru 0x2442bf (0x3242 bytes) 2024-09-16T11:44:29.605 INFO:teuthology.orchestra.run.smithi070.stdout:5618 punch from 0x393393 to 0x39f6c8, (0xc335 bytes) 2024-09-16T11:44:29.616 INFO:teuthology.orchestra.run.smithi070.stdout:5620 trunc from 0x7526b7 to 0x98ae604 2024-09-16T11:44:29.626 INFO:teuthology.orchestra.run.smithi070.stdout:5621 write 0x9cf6dd4 thru 0x9cfa3ba (0x35e7 bytes) 2024-09-16T11:44:29.636 INFO:teuthology.orchestra.run.smithi070.stdout:5623 write 0xb23b96d thru 0xb241487 (0x5b1b bytes) 2024-09-16T11:44:29.645 INFO:teuthology.orchestra.run.smithi070.stdout:5624 write 0xbe14e5d thru 0xbe2420f (0xf3b3 bytes) 2024-09-16T11:44:29.647 INFO:teuthology.orchestra.run.smithi070.stdout:5626 writesame 0x57db8a7 thru 0x57e71ca data_size 0x52(0xb924 bytes) 2024-09-16T11:44:29.648 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:29.648 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:29.649 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:29.646+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:29.651 INFO:teuthology.orchestra.run.smithi070.stdout:5627 write 0x8aa9ef8 thru 0x8ab060f (0x6718 bytes) 2024-09-16T11:44:29.654 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.655 INFO:teuthology.orchestra.run.smithi070.stdout:5634 write 0x3a0fb22 thru 0x3a1446e (0x494d bytes) 2024-09-16T11:44:29.657 INFO:teuthology.orchestra.run.smithi070.stdout:5635 read 0x3397a03 thru 0x339a634 (0x2c32 bytes) 2024-09-16T11:44:29.657 INFO:teuthology.orchestra.run.smithi070.stdout:5637 read 0x15b2c12 thru 0x15bf94e (0xcd3d bytes) 2024-09-16T11:44:29.667 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:29.668 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:29.668 INFO:teuthology.orchestra.run.smithi070.stdout:5638 trunc from 0xbe24210 to 0x32ff8f7 2024-09-16T11:44:29.681 INFO:teuthology.orchestra.run.smithi070.stdout:5640 read 0x12901c7 thru 0x129349a (0x32d4 bytes) 2024-09-16T11:44:29.682 INFO:teuthology.orchestra.run.smithi070.stdout:5642 punch from 0x177cc02 to 0x177e4e0, (0x18de bytes) 2024-09-16T11:44:29.686 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:29.686 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:29.687 INFO:teuthology.orchestra.run.smithi070.stdout:5643 trunc from 0x32ff8f7 to 0x7836d5d 2024-09-16T11:44:29.692 INFO:teuthology.orchestra.run.smithi070.stdout:5644 punch from 0x3cd5366 to 0x3cdfb12, (0xa7ac bytes) 2024-09-16T11:44:29.692 INFO:teuthology.orchestra.run.smithi070.stdout:5645 write 0x245317a thru 0x2457044 (0x3ecb bytes) 2024-09-16T11:44:29.695 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.699 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:29.699 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:29.699 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:29.702 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:29.703 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:29.708 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:29.709 INFO:teuthology.orchestra.run.smithi070.stdout:5647 writesame 0xe89b2d6 thru 0xe89c7b5 data_size 0x8(0x14e0 bytes) 2024-09-16T11:44:29.709 INFO:teuthology.orchestra.run.smithi070.stdout:5648 write 0xad19706 thru 0xad252a1 (0xbb9c bytes) 2024-09-16T11:44:29.711 INFO:teuthology.orchestra.run.smithi070.stdout:5649 read 0x75aed59 thru 0x75b4279 (0x5521 bytes) 2024-09-16T11:44:29.711 INFO:teuthology.orchestra.run.smithi070.stdout:5651 write 0x47ae718 thru 0x47bac1c (0xc505 bytes) 2024-09-16T11:44:29.714 INFO:teuthology.orchestra.run.smithi070.stdout:5653 writesame 0x52f2990 thru 0x5300974 data_size 0x1(0xdfe5 bytes) 2024-09-16T11:44:29.720 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:29.736 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:29.736 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:29.741 INFO:teuthology.orchestra.run.smithi070.stdout:5654 read 0x3d3a59c thru 0x3d3cb7a (0x25df bytes) 2024-09-16T11:44:29.741 INFO:teuthology.orchestra.run.smithi070.stdout:5657 write 0x6e4557 thru 0x6f052c (0xbfd6 bytes) 2024-09-16T11:44:29.743 INFO:teuthology.orchestra.run.smithi070.stdout:5658 read 0x1bac54b thru 0x1bb86ca (0xc180 bytes) 2024-09-16T11:44:29.743 INFO:teuthology.orchestra.run.smithi070.stdout:5660 write 0xa93b513 thru 0xa94149c (0x5f8a bytes) 2024-09-16T11:44:29.746 INFO:teuthology.orchestra.run.smithi070.stdout:5662 read 0xd0cb201 thru 0xd0d7173 (0xbf73 bytes) 2024-09-16T11:44:29.746 INFO:teuthology.orchestra.run.smithi070.stdout:5663 read 0x533059f thru 0x533568c (0x50ee bytes) 2024-09-16T11:44:29.746 INFO:teuthology.orchestra.run.smithi070.stdout:5665 punch from 0x997f038 to 0x998d895, (0xe85d bytes) 2024-09-16T11:44:29.746 INFO:teuthology.orchestra.run.smithi070.stdout:5666 read 0x45fc33e thru 0x4604286 (0x7f49 bytes) 2024-09-16T11:44:29.746 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.747 INFO:teuthology.orchestra.run.smithi070.stdout:5668 punch from 0x3109873 to 0x3118e4a, (0xf5d7 bytes) 2024-09-16T11:44:29.749 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:29.748+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:29.753 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:29.753 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:29.757 INFO:teuthology.orchestra.run.smithi070.stdout:5669 trunc from 0xe89c7b6 to 0x56b6c4e 2024-09-16T11:44:29.773 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:29.773 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:29.775 INFO:teuthology.orchestra.run.smithi070.stdout:5670 write 0x7c3acaf thru 0x7c4021b (0x556d bytes) 2024-09-16T11:44:29.778 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.786 INFO:teuthology.orchestra.run.smithi070.stdout:5672 trunc from 0x7c4021c to 0xd58caa9 2024-09-16T11:44:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:5673 punch from 0x979651d to 0x979a282, (0x3d65 bytes) 2024-09-16T11:44:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:5674 read 0x81c79a2 thru 0x81ca417 (0x2a76 bytes) 2024-09-16T11:44:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:5677 punch from 0xb8955e9 to 0xb8995fa, (0x4011 bytes) 2024-09-16T11:44:29.791 INFO:teuthology.orchestra.run.smithi070.stdout:5678 writesame 0x4e6bdfc thru 0x4e6e6f1 data_size 0x31(0x28f6 bytes) 2024-09-16T11:44:29.793 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.793 INFO:teuthology.orchestra.run.smithi070.stdout:5682 punch from 0x96d3d1d to 0x96df992, (0xbc75 bytes) 2024-09-16T11:44:29.800 INFO:teuthology.orchestra.run.smithi070.stdout:5683 write 0xe27fca4 thru 0xe28e65e (0xe9bb bytes) 2024-09-16T11:44:29.801 INFO:teuthology.orchestra.run.smithi070.stdout:5684 write 0x86013ec thru 0x8609f18 (0x8b2d bytes) 2024-09-16T11:44:29.805 INFO:teuthology.orchestra.run.smithi070.stdout:5686 punch from 0x816a0e8 to 0x8170144, (0x605c bytes) 2024-09-16T11:44:29.805 INFO:teuthology.orchestra.run.smithi070.stdout:5687 writesame 0xc71c9e7 thru 0xc71fa96 data_size 0x4c(0x30b0 bytes) 2024-09-16T11:44:29.808 INFO:teuthology.orchestra.run.smithi070.stdout:5688 punch from 0x9308749 to 0x9310eca, (0x8781 bytes) 2024-09-16T11:44:29.808 INFO:teuthology.orchestra.run.smithi070.stdout:5689 read 0x9611e36 thru 0x961b381 (0x954c bytes) 2024-09-16T11:44:29.808 INFO:teuthology.orchestra.run.smithi070.stdout:5690 read 0x906a8e4 thru 0x9071798 (0x6eb5 bytes) 2024-09-16T11:44:29.808 INFO:teuthology.orchestra.run.smithi070.stdout:5693 punch from 0x826b08c to 0x82765f7, (0xb56b bytes) 2024-09-16T11:44:29.808 INFO:teuthology.orchestra.run.smithi070.stdout:5694 read 0x7c4c36a thru 0x7c4e2d9 (0x1f70 bytes) 2024-09-16T11:44:29.808 INFO:teuthology.orchestra.run.smithi070.stdout:5695 write 0x3b0d951 thru 0x3b1491c (0x6fcc bytes) 2024-09-16T11:44:29.810 INFO:teuthology.orchestra.run.smithi070.stdout:5698 read 0xc869f7f thru 0xc879319 (0xf39b bytes) 2024-09-16T11:44:29.810 INFO:teuthology.orchestra.run.smithi070.stdout:5699 read 0x55ec872 thru 0x55fbbdd (0xf36c bytes) 2024-09-16T11:44:29.810 INFO:teuthology.orchestra.run.smithi070.stdout:5700 write 0x919bd94 thru 0x919cafa (0xd67 bytes) 2024-09-16T11:44:29.812 INFO:teuthology.orchestra.run.smithi070.stdout:5702 write 0x26f53d0 thru 0x2705024 (0xfc55 bytes) 2024-09-16T11:44:29.815 INFO:teuthology.orchestra.run.smithi070.stdout:5703 read 0xc0d2ce1 thru 0xc0dc79c (0x9abc bytes) 2024-09-16T11:44:29.815 INFO:teuthology.orchestra.run.smithi070.stdout:5704 writesame 0x52874fa thru 0x528d15a data_size 0x1(0x5c61 bytes) 2024-09-16T11:44:29.826 INFO:teuthology.orchestra.run.smithi070.stdout:5705 read 0xac1e39e thru 0xac1f451 (0x10b4 bytes) 2024-09-16T11:44:29.827 INFO:teuthology.orchestra.run.smithi070.stdout:5708 read 0x6f3de05 thru 0x6f46b77 (0x8d73 bytes) 2024-09-16T11:44:29.827 INFO:teuthology.orchestra.run.smithi070.stdout:5709 write 0x88b2841 thru 0x88b80c6 (0x5886 bytes) 2024-09-16T11:44:29.828 INFO:teuthology.orchestra.run.smithi070.stdout:5710 read 0x3c25fe6 thru 0x3c2b8a8 (0x58c3 bytes) 2024-09-16T11:44:29.828 INFO:teuthology.orchestra.run.smithi070.stdout:5711 writesame 0x4dce1ca thru 0x4dd61be data_size 0x3d(0x7ff5 bytes) 2024-09-16T11:44:29.831 INFO:teuthology.orchestra.run.smithi070.stdout:5714 read 0x5781511 thru 0x57826eb (0x11db bytes) 2024-09-16T11:44:29.836 INFO:teuthology.orchestra.run.smithi070.stdout:5715 write 0xe8cc58a thru 0xe8d49ed (0x8464 bytes) 2024-09-16T11:44:29.839 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.843 INFO:teuthology.orchestra.run.smithi070.stdout:5717 write 0xe9fde00 thru 0xea0d547 (0xf748 bytes) 2024-09-16T11:44:29.850 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:29.848+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:29.851 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:29.851 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:29.851 INFO:teuthology.orchestra.run.smithi070.stdout:5718 trunc from 0xea0d548 to 0xa527ce1 2024-09-16T11:44:29.861 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:5724 read 0x1cd9963 thru 0x1cde8cc (0x4f6a bytes) 2024-09-16T11:44:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:5725 write 0x1e318a7 thru 0x1e3edab (0xd505 bytes) 2024-09-16T11:44:29.863 INFO:teuthology.orchestra.run.smithi070.stdout:5728 clone 36 order 19 su 32768 sc 11 2024-09-16T11:44:29.951 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:29.950+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:29.960 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:29.960 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:29.968 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:29.995 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:44:29.995 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:44:29.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:44:30.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:44:30.052 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:30.051+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:30.072 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:30.072 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:30.081 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:30.158 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-16T11:44:30.159 INFO:teuthology.orchestra.run.smithi070.stdout:5729 write 0x4838807 thru 0x4846924 (0xe11e bytes) 2024-09-16T11:44:30.165 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:30.164+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:30.168 INFO:teuthology.orchestra.run.smithi070.stdout:5731 read 0x188b8c4 thru 0x189aaef (0xf22c bytes) 2024-09-16T11:44:30.174 INFO:teuthology.orchestra.run.smithi070.stdout:5732 write 0xabc66f2 thru 0xabce288 (0x7b97 bytes) 2024-09-16T11:44:30.178 INFO:teuthology.orchestra.run.smithi070.stdout:5733 punch from 0x7c86dcf to 0x7c90581, (0x97b2 bytes) 2024-09-16T11:44:30.178 INFO:teuthology.orchestra.run.smithi070.stdout:5735 write 0x58a1723 thru 0x58af9c3 (0xe2a1 bytes) 2024-09-16T11:44:30.195 INFO:teuthology.orchestra.run.smithi070.stdout:5736 trunc from 0xabce289 to 0xc1b9bf 2024-09-16T11:44:30.222 INFO:teuthology.orchestra.run.smithi070.stdout:5737 punch from 0x2a5410 to 0x2a6f9e, (0x1b8e bytes) 2024-09-16T11:44:30.229 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:30.229 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:30.238 INFO:teuthology.orchestra.run.smithi070.stdout:5739 write 0x8e5edd1 thru 0x8e62969 (0x3b99 bytes) 2024-09-16T11:44:30.239 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:30.247 INFO:teuthology.orchestra.run.smithi070.stdout:5740 trunc from 0x8e6296a to 0xde335fc 2024-09-16T11:44:30.252 INFO:teuthology.orchestra.run.smithi070.stdout:5743 read 0xaf031a9 thru 0xaf0a016 (0x6e6e bytes) 2024-09-16T11:44:30.252 INFO:teuthology.orchestra.run.smithi070.stdout:5744 writesame 0xc66edd6 thru 0xc678a9f data_size 0x2(0x9cca bytes) 2024-09-16T11:44:30.263 INFO:teuthology.orchestra.run.smithi070.stdout:5746 read 0xd14c76f thru 0xd14de37 (0x16c9 bytes) 2024-09-16T11:44:30.263 INFO:teuthology.orchestra.run.smithi070.stdout:5748 write 0x6ed6981 thru 0x6ed7311 (0x991 bytes) 2024-09-16T11:44:30.266 INFO:teuthology.orchestra.run.smithi070.stdout:5749 write 0x6393f1d thru 0x6399cb7 (0x5d9b bytes) 2024-09-16T11:44:30.269 INFO:teuthology.orchestra.run.smithi070.stdout:5750 read 0x39d6b0 thru 0x39fed7 (0x2828 bytes) 2024-09-16T11:44:30.280 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:30.279+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:30.282 INFO:teuthology.orchestra.run.smithi070.stdout:5751 trunc from 0xde335fc to 0x3499b05 2024-09-16T11:44:30.305 INFO:teuthology.orchestra.run.smithi070.stdout:5752 write 0x78706a9 thru 0x787cc38 (0xc590 bytes) 2024-09-16T11:44:30.312 INFO:teuthology.orchestra.run.smithi070.stdout:5754 trunc from 0x787cc39 to 0x49cd337 2024-09-16T11:44:30.329 INFO:teuthology.orchestra.run.smithi070.stdout:5755 trunc from 0x49cd337 to 0xbda39bb 2024-09-16T11:44:30.334 INFO:teuthology.orchestra.run.smithi070.stdout:5756 write 0x1547002 thru 0x155022f (0x922e bytes) 2024-09-16T11:44:30.341 INFO:teuthology.orchestra.run.smithi070.stdout:5757 write 0xad6644a thru 0xad73870 (0xd427 bytes) 2024-09-16T11:44:30.347 INFO:teuthology.orchestra.run.smithi070.stdout:5759 write 0x1bdc4ca thru 0x1be9798 (0xd2cf bytes) 2024-09-16T11:44:30.355 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:30.354+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:30.358 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:30.359 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:44:30.360 INFO:teuthology.orchestra.run.smithi070.stdout:5760 trunc from 0xbda39bb to 0x9776ce0 2024-09-16T11:44:30.367 INFO:teuthology.orchestra.run.smithi070.stdout:5761 punch from 0x618948f to 0x6190c18, (0x7789 bytes) 2024-09-16T11:44:30.367 INFO:teuthology.orchestra.run.smithi070.stdout:5763 writesame 0x582f1d9 thru 0x583bbbe data_size 0x2b(0xc9e6 bytes) 2024-09-16T11:44:30.371 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-16T11:44:29.991143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1034298583,"num_objects":1603,"num_object_clones":251,"num_object_copies":3206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":42,"num_objects_unfound":0,"num_objects_dirty":1601,"num_whiteouts":920,"num_read":85459,"num_read_kb":386259,"num_write":14731,"num_write_kb":238090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":9964161,"num_keys_recovered":12,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12635,"ondisk_log_size":12635,"up":90,"acting":89,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2494956,"kb_used_data":37052,"kb_used_omap":13,"kb_used_meta":2457778,"kb_avail":752479764,"statfs":{"total":773094113280,"available":770539278336,"internally_reserved":0,"allocated":37941248,"data_stored":274830458,"data_compressed":10965673,"data_compressed_allocated":24719360,"data_compressed_original":268126818,"omap_allocated":13632,"internal_metadata":2516765376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":57782704,"num_objects":30,"num_object_clones":13,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":42,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":725,"num_read_kb":14876,"num_write":62,"num_write_kb":6703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1003804,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016358"},"pg_stats":[{"pgid":"4.1f","version":"73'243","reported_seq":2088,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.028947+0000","last_change":"2024-09-16T11:43:33.782279+0000","last_active":"2024-09-16T11:44:25.028947+0000","last_peered":"2024-09-16T11:44:25.028947+0000","last_clean":"2024-09-16T11:44:25.028947+0000","last_became_active":"2024-09-16T11:43:31.779428+0000","last_became_peered":"2024-09-16T11:43:31.779428+0000","last_unstale":"2024-09-16T11:44:25.028947+0000","last_undegraded":"2024-09-16T11:44:25.028947+0000","last_fullsized":"2024-09-16T11:44:25.028947+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:30.437698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090324899999999996,"stat_sum":{"num_bytes":21220983,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":16,"num_read":2651,"num_read_kb":10324,"num_write":241,"num_write_kb":6504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"74'340","reported_seq":1521,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.104401+0000","last_change":"2024-09-16T11:43:34.248024+0000","last_active":"2024-09-16T11:44:25.104401+0000","last_peered":"2024-09-16T11:44:25.104401+0000","last_clean":"2024-09-16T11:44:25.104401+0000","last_became_active":"2024-09-16T11:43:31.889706+0000","last_became_peered":"2024-09-16T11:43:31.889706+0000","last_unstale":"2024-09-16T11:44:25.104401+0000","last_undegraded":"2024-09-16T11:44:25.104401+0000","last_fullsized":"2024-09-16T11:44:25.104401+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:25.242116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027516299999999998,"stat_sum":{"num_bytes":29845012,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":1978,"num_read_kb":9457,"num_write":399,"num_write_kb":6916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"72'171","reported_seq":1234,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.976562+0000","last_change":"2024-09-16T11:43:33.777888+0000","last_active":"2024-09-16T11:44:25.976562+0000","last_peered":"2024-09-16T11:44:25.976562+0000","last_clean":"2024-09-16T11:44:25.976562+0000","last_became_active":"2024-09-16T11:43:31.775021+0000","last_became_peered":"2024-09-16T11:43:31.775021+0000","last_unstale":"2024-09-16T11:44:25.976562+0000","last_undegraded":"2024-09-16T11:44:25.976562+0000","last_fullsized":"2024-09-16T11:44:25.976562+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:50.045796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075167799999999998,"stat_sum":{"num_bytes":22664607,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":25,"num_read":1056,"num_read_kb":9229,"num_write":176,"num_write_kb":7332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"74'233","reported_seq":1840,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.064294+0000","last_change":"2024-09-16T11:43:33.772949+0000","last_active":"2024-09-16T11:44:25.064294+0000","last_peered":"2024-09-16T11:44:25.064294+0000","last_clean":"2024-09-16T11:44:25.064294+0000","last_became_active":"2024-09-16T11:43:31.773661+0000","last_became_peered":"2024-09-16T11:43:31.773661+0000","last_unstale":"2024-09-16T11:44:25.064294+0000","last_undegraded":"2024-09-16T11:44:25.064294+0000","last_fullsized":"2024-09-16T11:44:25.064294+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:36.062796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023334699999999999,"stat_sum":{"num_bytes":28129958,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":20,"num_read":1827,"num_read_kb":13030,"num_write":241,"num_write_kb":5618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"74'314","reported_seq":2111,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.079829+0000","last_change":"2024-09-16T11:43:33.773384+0000","last_active":"2024-09-16T11:44:25.079829+0000","last_peered":"2024-09-16T11:44:25.079829+0000","last_clean":"2024-09-16T11:44:25.079829+0000","last_became_active":"2024-09-16T11:43:31.774523+0000","last_became_peered":"2024-09-16T11:43:31.774523+0000","last_unstale":"2024-09-16T11:44:25.079829+0000","last_undegraded":"2024-09-16T11:44:25.079829+0000","last_fullsized":"2024-09-16T11:44:25.079829+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:15.703792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038485,"stat_sum":{"num_bytes":28395455,"num_objects":51,"num_object_clones":6,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":32,"num_read":2791,"num_read_kb":15082,"num_write":275,"num_write_kb":4057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"74'461","reported_seq":2820,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.082543+0000","last_change":"2024-09-16T11:43:33.774141+0000","last_active":"2024-09-16T11:44:25.082543+0000","last_peered":"2024-09-16T11:44:25.082543+0000","last_clean":"2024-09-16T11:44:25.082543+0000","last_became_active":"2024-09-16T11:43:31.774794+0000","last_became_peered":"2024-09-16T11:43:31.774794+0000","last_unstale":"2024-09-16T11:44:25.082543+0000","last_undegraded":"2024-09-16T11:44:25.082543+0000","last_fullsized":"2024-09-16T11:44:25.082543+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:06:57.981702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010218040000000001,"stat_sum":{"num_bytes":23268620,"num_objects":64,"num_object_clones":10,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":38,"num_read":3563,"num_read_kb":10093,"num_write":589,"num_write_kb":7582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"72'580","reported_seq":1151,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.975972+0000","last_change":"2024-09-16T11:43:33.777212+0000","last_active":"2024-09-16T11:44:25.975972+0000","last_peered":"2024-09-16T11:44:25.975972+0000","last_clean":"2024-09-16T11:44:25.975972+0000","last_became_active":"2024-09-16T11:43:31.769719+0000","last_became_peered":"2024-09-16T11:43:31.769719+0000","last_unstale":"2024-09-16T11:44:25.975972+0000","last_undegraded":"2024-09-16T11:44:25.975972+0000","last_fullsized":"2024-09-16T11:44:25.975972+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:06.911046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035706099999999999,"stat_sum":{"num_bytes":27930786,"num_objects":45,"num_object_clones":7,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":23,"num_read":2376,"num_read_kb":4282,"num_write":797,"num_write_kb":2258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"74'163","reported_seq":1157,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.087704+0000","last_change":"2024-09-16T11:43:33.774063+0000","last_active":"2024-09-16T11:44:25.087704+0000","last_peered":"2024-09-16T11:44:25.087704+0000","last_clean":"2024-09-16T11:44:25.087704+0000","last_became_active":"2024-09-16T11:43:31.776951+0000","last_became_peered":"2024-09-16T11:43:31.776951+0000","last_unstale":"2024-09-16T11:44:25.087704+0000","last_undegraded":"2024-09-16T11:44:25.087704+0000","last_fullsized":"2024-09-16T11:44:25.087704+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:28:39.026621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094366700000000003,"stat_sum":{"num_bytes":27102422,"num_objects":35,"num_object_clones":3,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":22,"num_read":1324,"num_read_kb":6834,"num_write":141,"num_write_kb":5365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"74'403","reported_seq":1751,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.976146+0000","last_change":"2024-09-16T11:43:33.777913+0000","last_active":"2024-09-16T11:44:25.976146+0000","last_peered":"2024-09-16T11:44:25.976146+0000","last_clean":"2024-09-16T11:44:25.976146+0000","last_became_active":"2024-09-16T11:43:31.889807+0000","last_became_peered":"2024-09-16T11:43:31.889807+0000","last_unstale":"2024-09-16T11:44:25.976146+0000","last_undegraded":"2024-09-16T11:44:25.976146+0000","last_fullsized":"2024-09-16T11:44:25.976146+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:30.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071799400000000001,"stat_sum":{"num_bytes":55385966,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":29,"num_read":3435,"num_read_kb":13046,"num_write":458,"num_write_kb":7885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"74'351","reported_seq":1616,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.204154+0000","last_change":"2024-09-16T11:43:34.248043+0000","last_active":"2024-09-16T11:44:25.204154+0000","last_peered":"2024-09-16T11:44:25.204154+0000","last_clean":"2024-09-16T11:44:25.204154+0000","last_became_active":"2024-09-16T11:43:31.776111+0000","last_became_peered":"2024-09-16T11:43:31.776111+0000","last_unstale":"2024-09-16T11:44:25.204154+0000","last_undegraded":"2024-09-16T11:44:25.204154+0000","last_fullsized":"2024-09-16T11:44:25.204154+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:21:28.658052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024883199999999999,"stat_sum":{"num_bytes":28052806,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":29,"num_read":2052,"num_read_kb":16791,"num_write":341,"num_write_kb":12034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"72'547","reported_seq":2369,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:26.016405+0000","last_change":"2024-09-16T11:43:33.785207+0000","last_active":"2024-09-16T11:44:26.016405+0000","last_peered":"2024-09-16T11:44:26.016405+0000","last_clean":"2024-09-16T11:44:26.016405+0000","last_became_active":"2024-09-16T11:43:31.889629+0000","last_became_peered":"2024-09-16T11:43:31.889629+0000","last_unstale":"2024-09-16T11:44:26.016405+0000","last_undegraded":"2024-09-16T11:44:26.016405+0000","last_fullsized":"2024-09-16T11:44:26.016405+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:53.124562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022324199999999999,"stat_sum":{"num_bytes":47626431,"num_objects":68,"num_object_clones":11,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":36,"num_read":3635,"num_read_kb":26167,"num_write":710,"num_write_kb":14999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"76'573","reported_seq":1462,"reported_epoch":76,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T11:44:27.686343+0000","last_change":"2024-09-16T11:44:27.664127+0000","last_active":"2024-09-16T11:44:27.686343+0000","last_peered":"2024-09-16T11:44:27.686343+0000","last_clean":"2024-09-16T11:44:24.088434+0000","last_became_active":"2024-09-16T11:44:27.661902+0000","last_became_peered":"2024-09-16T11:44:27.661902+0000","last_unstale":"2024-09-16T11:44:27.686343+0000","last_undegraded":"2024-09-16T11:44:27.686343+0000","last_fullsized":"2024-09-16T11:44:26.982631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:15.756046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085420199999999998,"stat_sum":{"num_bytes":35686271,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":42,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":2795,"num_read_kb":13069,"num_write":845,"num_write_kb":8912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1003804,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":42},{"shards":"3","objects":8}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"72'431","reported_seq":1957,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T11:44:23.972026+0000","last_change":"2024-09-16T11:43:33.773601+0000","last_active":"2024-09-16T11:44:23.972026+0000","last_peered":"2024-09-16T11:44:23.972026+0000","last_clean":"2024-09-16T11:44:23.972026+0000","last_became_active":"2024-09-16T11:43:31.776781+0000","last_became_peered":"2024-09-16T11:43:31.776781+0000","last_unstale":"2024-09-16T11:44:23.972026+0000","last_undegraded":"2024-09-16T11:44:23.972026+0000","last_fullsized":"2024-09-16T11:44:23.972026+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:14.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056674299999999998,"stat_sum":{"num_bytes":37751858,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":33,"num_read":3360,"num_read_kb":14690,"num_write":516,"num_write_kb":10489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"74'244","reported_seq":1576,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.978313+0000","last_change":"2024-09-16T11:43:33.773531+0000","last_active":"2024-09-16T11:44:25.978313+0000","last_peered":"2024-09-16T11:44:25.978313+0000","last_clean":"2024-09-16T11:44:25.978313+0000","last_became_active":"2024-09-16T11:43:31.780164+0000","last_became_peered":"2024-09-16T11:43:31.780164+0000","last_unstale":"2024-09-16T11:44:25.978313+0000","last_undegraded":"2024-09-16T11:44:25.978313+0000","last_fullsized":"2024-09-16T11:44:25.978313+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:56.846891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033807800000000001,"stat_sum":{"num_bytes":20555568,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":18,"num_read":1601,"num_read_kb":8695,"num_write":258,"num_write_kb":4840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"72'675","reported_seq":1704,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.975923+0000","last_change":"2024-09-16T11:43:33.777531+0000","last_active":"2024-09-16T11:44:25.975923+0000","last_peered":"2024-09-16T11:44:25.975923+0000","last_clean":"2024-09-16T11:44:25.975923+0000","last_became_active":"2024-09-16T11:43:31.889754+0000","last_became_peered":"2024-09-16T11:43:31.889754+0000","last_unstale":"2024-09-16T11:44:25.975923+0000","last_undegraded":"2024-09-16T11:44:25.975923+0000","last_fullsized":"2024-09-16T11:44:25.975923+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:54.752770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062624299999999996,"stat_sum":{"num_bytes":22396270,"num_objects":61,"num_object_clones":6,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":44,"num_read":3736,"num_read_kb":16220,"num_write":955,"num_write_kb":7833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"74'290","reported_seq":1431,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.976171+0000","last_change":"2024-09-16T11:43:33.777407+0000","last_active":"2024-09-16T11:44:25.976171+0000","last_peered":"2024-09-16T11:44:25.976171+0000","last_clean":"2024-09-16T11:44:25.976171+0000","last_became_active":"2024-09-16T11:43:31.775962+0000","last_became_peered":"2024-09-16T11:43:31.775962+0000","last_unstale":"2024-09-16T11:44:25.976171+0000","last_undegraded":"2024-09-16T11:44:25.976171+0000","last_fullsized":"2024-09-16T11:44:25.976171+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:46.492489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062432399999999999,"stat_sum":{"num_bytes":16165986,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":31,"num_read":2556,"num_read_kb":8682,"num_write":269,"num_write_kb":1911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T11:44:23.969606+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:44:23.969606+0000","last_peered":"2024-09-16T11:44:23.969606+0000","last_clean":"2024-09-16T11:44:23.969606+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:44:23.969606+0000","last_undegraded":"2024-09-16T11:44:23.969606+0000","last_fullsized":"2024-09-16T11:44:23.969606+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"73'282","reported_seq":1332,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T11:44:24.193083+0000","last_change":"2024-09-16T11:43:33.773359+0000","last_active":"2024-09-16T11:44:24.193083+0000","last_peered":"2024-09-16T11:44:24.193083+0000","last_clean":"2024-09-16T11:44:24.193083+0000","last_became_active":"2024-09-16T11:43:31.777591+0000","last_became_peered":"2024-09-16T11:43:31.777591+0000","last_unstale":"2024-09-16T11:44:24.193083+0000","last_undegraded":"2024-09-16T11:44:24.193083+0000","last_fullsized":"2024-09-16T11:44:24.193083+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:00:39.586913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037893300000000003,"stat_sum":{"num_bytes":22585392,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":24,"num_read":2192,"num_read_kb":8156,"num_write":246,"num_write_kb":9407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":70,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.978454+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:44:25.978454+0000","last_peered":"2024-09-16T11:44:25.978454+0000","last_clean":"2024-09-16T11:44:25.978454+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:44:25.978454+0000","last_undegraded":"2024-09-16T11:44:25.978454+0000","last_fullsized":"2024-09-16T11:44:25.978454+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"72'254","reported_seq":2597,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.976247+0000","last_change":"2024-09-16T11:43:33.777644+0000","last_active":"2024-09-16T11:44:25.976247+0000","last_peered":"2024-09-16T11:44:25.976247+0000","last_clean":"2024-09-16T11:44:25.976247+0000","last_became_active":"2024-09-16T11:43:31.892159+0000","last_became_peered":"2024-09-16T11:43:31.892159+0000","last_unstale":"2024-09-16T11:44:25.976247+0000","last_undegraded":"2024-09-16T11:44:25.976247+0000","last_fullsized":"2024-09-16T11:44:25.976247+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:42:14.042476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062271400000000001,"stat_sum":{"num_bytes":43816440,"num_objects":57,"num_object_clones":10,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":31,"num_read":3064,"num_read_kb":17551,"num_write":226,"num_write_kb":7824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.978232+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:44:25.978232+0000","last_peered":"2024-09-16T11:44:25.978232+0000","last_clean":"2024-09-16T11:44:25.978232+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:44:25.978232+0000","last_undegraded":"2024-09-16T11:44:25.978232+0000","last_fullsized":"2024-09-16T11:44:25.978232+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"74'313","reported_seq":652,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.204493+0000","last_change":"2024-09-16T11:43:34.247771+0000","last_active":"2024-09-16T11:44:25.204493+0000","last_peered":"2024-09-16T11:44:25.204493+0000","last_clean":"2024-09-16T11:44:25.204493+0000","last_became_active":"2024-09-16T11:43:31.777896+0000","last_became_peered":"2024-09-16T11:43:31.777896+0000","last_unstale":"2024-09-16T11:44:25.204493+0000","last_undegraded":"2024-09-16T11:44:25.204493+0000","last_fullsized":"2024-09-16T11:44:25.204493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:35.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026844500000000002,"stat_sum":{"num_bytes":46146576,"num_objects":42,"num_object_clones":6,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":24,"num_read":452,"num_read_kb":5544,"num_write":321,"num_write_kb":6695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":70,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.978882+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:44:25.978882+0000","last_peered":"2024-09-16T11:44:25.978882+0000","last_clean":"2024-09-16T11:44:25.978882+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:44:25.978882+0000","last_undegraded":"2024-09-16T11:44:25.978882+0000","last_fullsized":"2024-09-16T11:44:25.978882+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"71'320","reported_seq":740,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.978330+0000","last_change":"2024-09-16T11:43:33.774367+0000","last_active":"2024-09-16T11:44:25.978330+0000","last_peered":"2024-09-16T11:44:25.978330+0000","last_clean":"2024-09-16T11:44:25.978330+0000","last_became_active":"2024-09-16T11:43:31.778506+0000","last_became_peered":"2024-09-16T11:43:31.778506+0000","last_unstale":"2024-09-16T11:44:25.978330+0000","last_undegraded":"2024-09-16T11:44:25.978330+0000","last_fullsized":"2024-09-16T11:44:25.978330+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:12.232493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010404660000000001,"stat_sum":{"num_bytes":5690215,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":1354,"num_read_kb":3456,"num_write":453,"num_write_kb":1994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":61,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T11:44:23.972089+0000","last_change":"2024-09-16T11:43:22.438212+0000","last_active":"2024-09-16T11:44:23.972089+0000","last_peered":"2024-09-16T11:44:23.972089+0000","last_clean":"2024-09-16T11:44:23.972089+0000","last_became_active":"2024-09-16T11:43:22.437868+0000","last_became_peered":"2024-09-16T11:43:22.437868+0000","last_unstale":"2024-09-16T11:44:23.972089+0000","last_undegraded":"2024-09-16T11:44:23.972089+0000","last_fullsized":"2024-09-16T11:44:23.972089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:35.498933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":72,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:26.016490+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:44:26.016490+0000","last_peered":"2024-09-16T11:44:26.016490+0000","last_clean":"2024-09-16T11:44:26.016490+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:44:26.016490+0000","last_undegraded":"2024-09-16T11:44:26.016490+0000","last_fullsized":"2024-09-16T11:44:26.016490+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"72'648","reported_seq":2184,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.978852+0000","last_change":"2024-09-16T11:43:33.774349+0000","last_active":"2024-09-16T11:44:25.978852+0000","last_peered":"2024-09-16T11:44:25.978852+0000","last_clean":"2024-09-16T11:44:25.978852+0000","last_became_active":"2024-09-16T11:43:31.777796+0000","last_became_peered":"2024-09-16T11:43:31.777796+0000","last_unstale":"2024-09-16T11:44:25.978852+0000","last_undegraded":"2024-09-16T11:44:25.978852+0000","last_fullsized":"2024-09-16T11:44:25.978852+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:33.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033368800000000001,"stat_sum":{"num_bytes":41535244,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":3932,"num_read_kb":14971,"num_write":851,"num_write_kb":10037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.978268+0000","last_change":"2024-09-16T11:43:22.440136+0000","last_active":"2024-09-16T11:44:25.978268+0000","last_peered":"2024-09-16T11:44:25.978268+0000","last_clean":"2024-09-16T11:44:25.978268+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:44:25.978268+0000","last_undegraded":"2024-09-16T11:44:25.978268+0000","last_fullsized":"2024-09-16T11:44:25.978268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:18.461948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T11:44:23.968767+0000","last_change":"2024-09-16T11:43:16.732834+0000","last_active":"2024-09-16T11:44:23.968767+0000","last_peered":"2024-09-16T11:44:23.968767+0000","last_clean":"2024-09-16T11:44:23.968767+0000","last_became_active":"2024-09-16T11:43:14.724728+0000","last_became_peered":"2024-09-16T11:43:14.724728+0000","last_unstale":"2024-09-16T11:44:23.968767+0000","last_undegraded":"2024-09-16T11:44:23.968767+0000","last_fullsized":"2024-09-16T11:44:23.968767+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:47.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"71'450","reported_seq":2884,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.978454+0000","last_change":"2024-09-16T11:43:33.774126+0000","last_active":"2024-09-16T11:44:25.978454+0000","last_peered":"2024-09-16T11:44:25.978454+0000","last_clean":"2024-09-16T11:44:25.978454+0000","last_became_active":"2024-09-16T11:43:31.777967+0000","last_became_peered":"2024-09-16T11:43:31.777967+0000","last_unstale":"2024-09-16T11:44:25.978454+0000","last_undegraded":"2024-09-16T11:44:25.978454+0000","last_fullsized":"2024-09-16T11:44:25.978454+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:43.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000373488,"stat_sum":{"num_bytes":54804694,"num_objects":60,"num_object_clones":12,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":31,"num_read":3375,"num_read_kb":18365,"num_write":555,"num_write_kb":6794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"56'1","reported_seq":61,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T11:44:23.975206+0000","last_change":"2024-09-16T11:43:22.437965+0000","last_active":"2024-09-16T11:44:23.975206+0000","last_peered":"2024-09-16T11:44:23.975206+0000","last_clean":"2024-09-16T11:44:23.975206+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:44:23.975206+0000","last_undegraded":"2024-09-16T11:44:23.975206+0000","last_fullsized":"2024-09-16T11:44:23.975206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:52.884360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":70,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.978249+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:44:25.978249+0000","last_peered":"2024-09-16T11:44:25.978249+0000","last_clean":"2024-09-16T11:44:25.978249+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:44:25.978249+0000","last_undegraded":"2024-09-16T11:44:25.978249+0000","last_fullsized":"2024-09-16T11:44:25.978249+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"74'265","reported_seq":1268,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.041695+0000","last_change":"2024-09-16T11:43:34.248254+0000","last_active":"2024-09-16T11:44:25.041695+0000","last_peered":"2024-09-16T11:44:25.041695+0000","last_clean":"2024-09-16T11:44:25.041695+0000","last_became_active":"2024-09-16T11:43:31.776645+0000","last_became_peered":"2024-09-16T11:43:31.776645+0000","last_unstale":"2024-09-16T11:44:25.041695+0000","last_undegraded":"2024-09-16T11:44:25.041695+0000","last_fullsized":"2024-09-16T11:44:25.041695+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:53.198323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038670399999999998,"stat_sum":{"num_bytes":59272607,"num_objects":58,"num_object_clones":13,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":31,"num_read":1407,"num_read_kb":8958,"num_write":260,"num_write_kb":9853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":62,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.976115+0000","last_change":"2024-09-16T11:43:22.587076+0000","last_active":"2024-09-16T11:44:25.976115+0000","last_peered":"2024-09-16T11:44:25.976115+0000","last_clean":"2024-09-16T11:44:25.976115+0000","last_became_active":"2024-09-16T11:43:22.586861+0000","last_became_peered":"2024-09-16T11:43:22.586861+0000","last_unstale":"2024-09-16T11:44:25.976115+0000","last_undegraded":"2024-09-16T11:44:25.976115+0000","last_fullsized":"2024-09-16T11:44:25.976115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:07:45.296752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":71,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:26.016468+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:44:26.016468+0000","last_peered":"2024-09-16T11:44:26.016468+0000","last_clean":"2024-09-16T11:44:26.016468+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:44:26.016468+0000","last_undegraded":"2024-09-16T11:44:26.016468+0000","last_fullsized":"2024-09-16T11:44:26.016468+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":157,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.978261+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:44:25.978261+0000","last_peered":"2024-09-16T11:44:25.978261+0000","last_clean":"2024-09-16T11:44:25.978261+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:44:25.978261+0000","last_undegraded":"2024-09-16T11:44:25.978261+0000","last_fullsized":"2024-09-16T11:44:25.978261+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"72'430","reported_seq":3019,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T11:44:24.088302+0000","last_change":"2024-09-16T11:43:33.781611+0000","last_active":"2024-09-16T11:44:24.088302+0000","last_peered":"2024-09-16T11:44:24.088302+0000","last_clean":"2024-09-16T11:44:24.088302+0000","last_became_active":"2024-09-16T11:43:31.782071+0000","last_became_peered":"2024-09-16T11:43:31.782071+0000","last_unstale":"2024-09-16T11:44:24.088302+0000","last_undegraded":"2024-09-16T11:44:24.088302+0000","last_fullsized":"2024-09-16T11:44:24.088302+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:15.516215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033637000000000001,"stat_sum":{"num_bytes":66692159,"num_objects":65,"num_object_clones":12,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":29,"num_read":3570,"num_read_kb":24178,"num_write":476,"num_write_kb":7856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"72'536","reported_seq":3307,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.978180+0000","last_change":"2024-09-16T11:44:10.921411+0000","last_active":"2024-09-16T11:44:25.978180+0000","last_peered":"2024-09-16T11:44:25.978180+0000","last_clean":"2024-09-16T11:44:25.978180+0000","last_became_active":"2024-09-16T11:44:10.920131+0000","last_became_peered":"2024-09-16T11:44:10.920131+0000","last_unstale":"2024-09-16T11:44:25.978180+0000","last_undegraded":"2024-09-16T11:44:25.978180+0000","last_fullsized":"2024-09-16T11:44:25.978180+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:55:10.158414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398253,"stat_sum":{"num_bytes":54975502,"num_objects":72,"num_object_clones":11,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":3998,"num_read_kb":33457,"num_write":615,"num_write_kb":18511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":8960357,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"74'591","reported_seq":2868,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.047814+0000","last_change":"2024-09-16T11:43:33.773950+0000","last_active":"2024-09-16T11:44:25.047814+0000","last_peered":"2024-09-16T11:44:25.047814+0000","last_clean":"2024-09-16T11:44:25.047814+0000","last_became_active":"2024-09-16T11:43:31.775309+0000","last_became_peered":"2024-09-16T11:43:31.775309+0000","last_unstale":"2024-09-16T11:44:25.047814+0000","last_undegraded":"2024-09-16T11:44:25.047814+0000","last_fullsized":"2024-09-16T11:44:25.047814+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:23.106465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086187000000000004,"stat_sum":{"num_bytes":35801212,"num_objects":59,"num_object_clones":9,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":34,"num_read":6435,"num_read_kb":7780,"num_write":573,"num_write_kb":5222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"74'491","reported_seq":1333,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.028902+0000","last_change":"2024-09-16T11:43:33.781852+0000","last_active":"2024-09-16T11:44:25.028902+0000","last_peered":"2024-09-16T11:44:25.028902+0000","last_clean":"2024-09-16T11:44:25.028902+0000","last_became_active":"2024-09-16T11:43:31.779384+0000","last_became_peered":"2024-09-16T11:43:31.779384+0000","last_unstale":"2024-09-16T11:44:25.028902+0000","last_undegraded":"2024-09-16T11:44:25.028902+0000","last_fullsized":"2024-09-16T11:44:25.028902+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:34.933146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029212900000000002,"stat_sum":{"num_bytes":18297055,"num_objects":57,"num_object_clones":7,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":37,"num_read":2188,"num_read_kb":8819,"num_write":555,"num_write_kb":5957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"72'300","reported_seq":1240,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:24.976847+0000","last_change":"2024-09-16T11:43:34.247758+0000","last_active":"2024-09-16T11:44:24.976847+0000","last_peered":"2024-09-16T11:44:24.976847+0000","last_clean":"2024-09-16T11:44:24.976847+0000","last_became_active":"2024-09-16T11:43:31.775335+0000","last_became_peered":"2024-09-16T11:43:31.775335+0000","last_unstale":"2024-09-16T11:44:24.976847+0000","last_undegraded":"2024-09-16T11:44:24.976847+0000","last_fullsized":"2024-09-16T11:44:24.976847+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:07.521713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026563699999999999,"stat_sum":{"num_bytes":29145232,"num_objects":43,"num_object_clones":4,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":25,"num_read":2071,"num_read_kb":7915,"num_write":355,"num_write_kb":9408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"72'359","reported_seq":1523,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T11:44:24.088337+0000","last_change":"2024-09-16T11:43:33.773570+0000","last_active":"2024-09-16T11:44:24.088337+0000","last_peered":"2024-09-16T11:44:24.088337+0000","last_clean":"2024-09-16T11:44:24.088337+0000","last_became_active":"2024-09-16T11:43:31.774601+0000","last_became_peered":"2024-09-16T11:43:31.774601+0000","last_unstale":"2024-09-16T11:44:24.088337+0000","last_undegraded":"2024-09-16T11:44:24.088337+0000","last_fullsized":"2024-09-16T11:44:24.088337+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:12.584420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055828300000000002,"stat_sum":{"num_bytes":13260014,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":29,"num_read":2636,"num_read_kb":4041,"num_write":336,"num_write_kb":2260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"74'546","reported_seq":1577,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.057394+0000","last_change":"2024-09-16T11:43:33.773852+0000","last_active":"2024-09-16T11:44:25.057394+0000","last_peered":"2024-09-16T11:44:25.057394+0000","last_clean":"2024-09-16T11:44:25.057394+0000","last_became_active":"2024-09-16T11:43:31.772506+0000","last_became_peered":"2024-09-16T11:43:31.772506+0000","last_unstale":"2024-09-16T11:44:25.057394+0000","last_undegraded":"2024-09-16T11:44:25.057394+0000","last_fullsized":"2024-09-16T11:44:25.057394+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:36.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076283899999999997,"stat_sum":{"num_bytes":23353889,"num_objects":44,"num_object_clones":5,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":28,"num_read":3104,"num_read_kb":14354,"num_write":755,"num_write_kb":9867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"74'448","reported_seq":1253,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.205104+0000","last_change":"2024-09-16T11:43:33.773770+0000","last_active":"2024-09-16T11:44:25.205104+0000","last_peered":"2024-09-16T11:44:25.205104+0000","last_clean":"2024-09-16T11:44:25.205104+0000","last_became_active":"2024-09-16T11:43:31.779613+0000","last_became_peered":"2024-09-16T11:43:31.779613+0000","last_unstale":"2024-09-16T11:44:25.205104+0000","last_undegraded":"2024-09-16T11:44:25.205104+0000","last_fullsized":"2024-09-16T11:44:25.205104+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:40.995663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000720899,"stat_sum":{"num_bytes":38934270,"num_objects":59,"num_object_clones":11,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":30,"num_read":1957,"num_read_kb":8996,"num_write":546,"num_write_kb":4901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"74'306","reported_seq":1460,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T11:44:25.978426+0000","last_change":"2024-09-16T11:43:33.773914+0000","last_active":"2024-09-16T11:44:25.978426+0000","last_peered":"2024-09-16T11:44:25.978426+0000","last_clean":"2024-09-16T11:44:25.978426+0000","last_became_active":"2024-09-16T11:43:31.776132+0000","last_became_peered":"2024-09-16T11:43:31.776132+0000","last_unstale":"2024-09-16T11:44:25.978426+0000","last_undegraded":"2024-09-16T11:44:25.978426+0000","last_fullsized":"2024-09-16T11:44:25.978426+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:53.489198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023678600000000001,"stat_sum":{"num_bytes":7218696,"num_objects":44,"num_object_clones":6,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":27,"num_read":2922,"num_read_kb":3972,"num_write":294,"num_write_kb":5339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1033708196,"num_objects":1597,"num_object_clones":251,"num_object_copies":3194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":42,"num_objects_unfound":0,"num_objects_dirty":1597,"num_whiteouts":918,"num_read":85393,"num_read_kb":386204,"num_write":14628,"num_write_kb":236460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":9964161,"num_keys_recovered":12,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":31080448,"data_stored":271365767,"data_compressed":11022616,"data_compressed_allocated":24784896,"data_compressed_original":268012130,"omap_allocated":0,"internal_metadata":0},"log_size":12558,"ondisk_log_size":12558,"up":64,"acting":63,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315228,"kb_used_data":5388,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94056612,"statfs":{"total":96636764160,"available":96313970688,"internally_reserved":0,"allocated":5517312,"data_stored":45297023,"data_compressed":1473265,"data_compressed_allocated":3887104,"data_compressed_original":44465069,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.498},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.47399999999999998}]},{"osd":1,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":3.2469999999999999,"5min":3.2469999999999999,"15min":3.2469999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.34799999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.51000000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.5640000000000001,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.314},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.46000000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.54000000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48799999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313684,"kb_used_data":3844,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058156,"statfs":{"total":96636764160,"available":96315551744,"internally_reserved":0,"allocated":3936256,"data_stored":27549524,"data_compressed":1276190,"data_compressed_allocated":2416640,"data_compressed_original":26764282,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.28199999999999997}]},{"osd":1,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.34599999999999997}]},{"osd":2,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53100000000000003}]},{"osd":3,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.46700000000000003}]},{"osd":4,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.378},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.43099999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.41599999999999998}]},{"osd":7,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.496},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.48399999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311292,"kb_used_data":5616,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94060548,"statfs":{"total":96636764160,"available":96318001152,"internally_reserved":0,"allocated":5750784,"data_stored":41834837,"data_compressed":1882982,"data_compressed_allocated":3874816,"data_compressed_original":40877346,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.17299999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.187},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.32700000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.377}]},{"osd":4,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.36699999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.314},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.42399999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.439}]},{"osd":7,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.39500000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312724,"kb_used_data":7052,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94059116,"statfs":{"total":96636764160,"available":96316534784,"internally_reserved":0,"allocated":7221248,"data_stored":53601495,"data_compressed":2444158,"data_compressed_allocated":5054464,"data_compressed_original":52516319,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.24399999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.30099999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.32900000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.313},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40300000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.35099999999999998}]},{"osd":6,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.503},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.42799999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.40300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313512,"kb_used_data":3668,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058328,"statfs":{"total":96636764160,"available":96315727872,"internally_reserved":0,"allocated":3756032,"data_stored":28710517,"data_compressed":786761,"data_compressed_allocated":2281472,"data_compressed_original":27983193,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.29499999999999998}]},{"osd":1,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.245}]},{"osd":2,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.375},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.38400000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.33600000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39800000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.34699999999999998}]},{"osd":7,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.36199999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302440,"kb_used_data":5088,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069400,"statfs":{"total":96636764160,"available":96327065600,"internally_reserved":0,"allocated":5210112,"data_stored":37339129,"data_compressed":1461455,"data_compressed_allocated":3309568,"data_compressed_original":36352903,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.39500000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.29899999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.26800000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.32400000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57699999999999996}]},{"osd":6,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.48099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312368,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059472,"statfs":{"total":96636764160,"available":96316899328,"internally_reserved":0,"allocated":2592768,"data_stored":14795209,"data_compressed":550730,"data_compressed_allocated":1331200,"data_compressed_original":14160416,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.186},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":1.006}]},{"osd":1,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.315},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.496}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.249}]},{"osd":4,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.55900000000000005}]},{"osd":5,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.39600000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.442},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.36699999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.629},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.44900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313708,"kb_used_data":3864,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058132,"statfs":{"total":96636764160,"available":96315527168,"internally_reserved":0,"allocated":3956736,"data_stored":25702724,"data_compressed":1090132,"data_compressed_allocated":2564096,"data_compressed_original":25007290,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.38400000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.40300000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.441}]},{"osd":4,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70399999999999996}]},{"osd":5,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.53100000000000003}]},{"osd":6,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.63}]},{"osd":7,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.68700000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3035136,"data_stored":24696165,"data_compressed":1084976,"data_compressed_allocated":2527232,"data_compressed_original":24433850,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6328320,"data_stored":53172778,"data_compressed":2444158,"data_compressed_allocated":5054464,"data_compressed_original":52516319,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1744896,"data_stored":14385609,"data_compressed":550730,"data_compressed_allocated":1331200,"data_compressed_original":14160416,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4874240,"data_stored":41412379,"data_compressed":1882982,"data_compressed_allocated":3874816,"data_compressed_original":40877346,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4345856,"data_stored":36922938,"data_compressed":1461455,"data_compressed_allocated":3309568,"data_compressed_original":36352903,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3055616,"data_stored":27127048,"data_compressed":1276190,"data_compressed_allocated":2416640,"data_compressed_original":26764282,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2920448,"data_stored":28360848,"data_compressed":786797,"data_compressed_allocated":2285568,"data_compressed_original":28036441,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4775936,"data_stored":45288002,"data_compressed":1535328,"data_compressed_allocated":3985408,"data_compressed_original":44870573,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:44:30.375 INFO:teuthology.orchestra.run.smithi070.stdout:5765 writesame 0x5569e09 thru 0x5577976 data_size 0x1(0xdb6e bytes) 2024-09-16T11:44:30.389 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:30.389 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:30.391 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:30.391 INFO:teuthology.orchestra.run.smithi070.stdout:5768 write 0x2a8b511 thru 0x2a982df (0xcdcf bytes) 2024-09-16T11:44:30.396 INFO:teuthology.orchestra.run.smithi070.stdout:5769 clone 37 order 25 su 524288 sc 14 2024-09-16T11:44:30.399 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:30.456 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:30.455+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:30.536 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:30.537 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:30.544 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:30.558 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:30.556+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:30.643 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:30.643 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:30.651 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:30.695 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:30.693+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:30.749 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:30.750 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:30.758 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:30.760 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:30.759+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:30.861 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:30.859+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:30.895 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:30.896 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:30.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:30.962 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:30.960+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:30.996 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T11:44:31.004 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:31.004 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:31.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:31.055 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone36 from 0x9776ce0 (overlap 0xc1b9bf) to 0x135f5f 2024-09-16T11:44:31.099 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:31.097+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:31.150 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-16T11:44:31.164 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:31.162+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:31.177 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:31.178 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:31.190 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:31.275 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:31.273+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:31.331 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:31.332 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:31.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:31.346 INFO:teuthology.orchestra.run.smithi070.stdout:5770 write 0xe6f6250 thru 0xe6f895a (0x270b bytes) 2024-09-16T11:44:31.354 INFO:teuthology.orchestra.run.smithi070.stdout:5772 read 0x4bbb7e6 thru 0x4bc8bf3 (0xd40e bytes) 2024-09-16T11:44:31.354 INFO:teuthology.orchestra.run.smithi070.stdout:5773 read 0x13bc246 thru 0x13c8f00 (0xccbb bytes) 2024-09-16T11:44:31.354 INFO:teuthology.orchestra.run.smithi070.stdout:5776 write 0x51318d thru 0x522035 (0xeea9 bytes) 2024-09-16T11:44:31.361 INFO:teuthology.orchestra.run.smithi070.stdout:5777 write 0x4da0f96 thru 0x4daedf0 (0xde5b bytes) 2024-09-16T11:44:31.365 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:31.364+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:31.370 INFO:teuthology.orchestra.run.smithi070.stdout:5780 punch from 0xd6017b9 to 0xd60b17d, (0x99c4 bytes) 2024-09-16T11:44:31.370 INFO:teuthology.orchestra.run.smithi070.stdout:5781 write 0x2408891 thru 0x2408bd9 (0x349 bytes) 2024-09-16T11:44:31.377 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:31.377 INFO:teuthology.orchestra.run.smithi070.stdout:5783 writesame 0x7717c13 thru 0x771fa17 data_size 0x1(0x7e05 bytes) 2024-09-16T11:44:31.400 INFO:teuthology.orchestra.run.smithi070.stdout:5784 read 0x9fdd131 thru 0x9fecab4 (0xf984 bytes) 2024-09-16T11:44:31.400 INFO:teuthology.orchestra.run.smithi070.stdout:5786 punch from 0x20ade10 to 0x20b21f6, (0x43e6 bytes) 2024-09-16T11:44:31.415 INFO:teuthology.orchestra.run.smithi070.stdout:5788 trunc from 0xe6f895b to 0x194a701 2024-09-16T11:44:31.459 INFO:teuthology.orchestra.run.smithi070.stdout:5789 write 0x6827bcc thru 0x682c811 (0x4c46 bytes) 2024-09-16T11:44:31.460 INFO:teuthology.orchestra.run.smithi070.stdout:5793 write 0x35bca14 thru 0x35cc752 (0xfd3f bytes) 2024-09-16T11:44:31.461 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:31.461 INFO:teuthology.orchestra.run.smithi070.stdout:5797 read 0x4288f91 thru 0x428b19d (0x220d bytes) 2024-09-16T11:44:31.467 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:31.465+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:31.467 INFO:teuthology.orchestra.run.smithi070.stdout:5798 write 0x6e8287f thru 0x6e89cf2 (0x7474 bytes) 2024-09-16T11:44:31.468 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:31.468 INFO:teuthology.orchestra.run.smithi070.stdout:5802 writesame 0x535a775 thru 0x5363621 data_size 0x19(0x8ead bytes) 2024-09-16T11:44:31.470 INFO:teuthology.orchestra.run.smithi070.stdout:5804 read 0x187c911 thru 0x188b500 (0xebf0 bytes) 2024-09-16T11:44:31.470 INFO:teuthology.orchestra.run.smithi070.stdout:5805 write 0x4c426cc thru 0x4c4b185 (0x8aba bytes) 2024-09-16T11:44:31.471 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:31.471 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:31.472 INFO:teuthology.orchestra.run.smithi070.stdout:5806 write 0xa40b9a thru 0xa4ecaa (0xe111 bytes) 2024-09-16T11:44:31.473 INFO:teuthology.orchestra.run.smithi070.stdout:5807 read 0xce3aa3 thru 0xce3e55 (0x3b3 bytes) 2024-09-16T11:44:31.474 INFO:teuthology.orchestra.run.smithi070.stdout:5808 writesame 0x36e685b thru 0x36eda6a data_size 0x32(0x7210 bytes) 2024-09-16T11:44:31.483 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:31.484 INFO:teuthology.orchestra.run.smithi070.stdout:5809 writesame 0x86093db thru 0x860f78e data_size 0x12(0x63b4 bytes) 2024-09-16T11:44:31.484 INFO:teuthology.orchestra.run.smithi070.stdout:5810 clone 38 order 24 su 2097152 sc 7 2024-09-16T11:44:31.567 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:31.566+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:31.582 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:31.582 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:31.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:31.668 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:31.666+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:31.688 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:31.689 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:31.698 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:31.769 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:31.768+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:31.787 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:44:31.838 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:31.839 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:31.848 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:31.896 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:31.894+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:31.904 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:31.904 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:31.921 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:31.921 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:31.936 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:31.951 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:31.951 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:31.955 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:31.956 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:31.956 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:31.956 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:31.965 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:31.967 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:31.967 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:31.970 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:31.969+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:31.982 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:31.983 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:32.001 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:32.001 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:32.019 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:32.019 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:32.066 INFO:teuthology.orchestra.run.smithi070.stdout:truncating image image_client.0-clone37 from 0x860f78f (overlap 0x194a701) to 0x50ee33 2024-09-16T11:44:32.071 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:32.069+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:32.080 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:32.081 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:32.090 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:32.137 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-16T11:44:32.143 INFO:teuthology.orchestra.run.smithi070.stdout:5811 trunc from 0x860f78f to 0x65e498e 2024-09-16T11:44:32.171 INFO:teuthology.orchestra.run.smithi070.stdout:5812 trunc from 0x65e498e to 0x6c89218 2024-09-16T11:44:32.172 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:32.170+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:32.190 INFO:teuthology.orchestra.run.smithi070.stdout:5814 write 0xd3952de thru 0xd395dfc (0xb1f bytes) 2024-09-16T11:44:32.193 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.193 INFO:teuthology.orchestra.run.smithi070.stdout:5817 read 0xb0ce73f thru 0xb0d4633 (0x5ef5 bytes) 2024-09-16T11:44:32.193 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.193 INFO:teuthology.orchestra.run.smithi070.stdout:5819 writesame 0x6901702 thru 0x6904148 data_size 0x1(0x2a47 bytes) 2024-09-16T11:44:32.198 INFO:teuthology.orchestra.run.smithi070.stdout:5820 punch from 0xcfbcf18 to 0xcfcc43f, (0xf527 bytes) 2024-09-16T11:44:32.199 INFO:teuthology.orchestra.run.smithi070.stdout:5822 read 0x78c7117 thru 0x78cf8ae (0x8798 bytes) 2024-09-16T11:44:32.199 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.207 INFO:teuthology.orchestra.run.smithi070.stdout:5824 trunc from 0xd395dfd to 0x6564608 2024-09-16T11:44:32.231 INFO:teuthology.orchestra.run.smithi070.stdout:5826 write 0xbae39f3 thru 0xbae3be3 (0x1f1 bytes) 2024-09-16T11:44:32.235 INFO:teuthology.orchestra.run.smithi070.stdout:5827 write 0x3ac6dda thru 0x3ad5382 (0xe5a9 bytes) 2024-09-16T11:44:32.237 INFO:teuthology.orchestra.run.smithi070.stdout:5828 read 0x3818ef7 thru 0x38240c0 (0xb1ca bytes) 2024-09-16T11:44:32.242 INFO:teuthology.orchestra.run.smithi070.stdout:5829 write 0xc12229a thru 0xc123fad (0x1d14 bytes) 2024-09-16T11:44:32.245 INFO:teuthology.orchestra.run.smithi070.stdout:5831 write 0x2b76429 thru 0x2b78235 (0x1e0d bytes) 2024-09-16T11:44:32.252 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:32.252 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:32.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:32.252 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:32.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.253 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19545 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4415488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.253 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:32.253 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.725962611, 2024-09-16T11:44:32.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043429581000000002, 2024-09-16T11:44:32.253 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.253 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.253 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.253 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.253 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.253 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19545 2024-09-16T11:44:32.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.253 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:32.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:32.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523839+0000", 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0220000000000004e-06 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523845+0000", 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6500000000000001e-06 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523848+0000", 2024-09-16T11:44:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1879999999999998e-06 2024-09-16T11:44:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567092+0000", 2024-09-16T11:44:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043244145999999997 2024-09-16T11:44:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567176+0000", 2024-09-16T11:44:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3825999999999993e-05 2024-09-16T11:44:32.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567263+0000", 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7423999999999995e-05 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19546 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4734976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.725944771000002, 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043599231000000002, 2024-09-16T11:44:32.257 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.258 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.258 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.258 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.258 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.258 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19546 2024-09-16T11:44:32.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.258 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:32.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523954+0000", 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010217099999999999 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523957+0000", 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3270000000000001e-06 2024-09-16T11:44:32.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523966+0000", 2024-09-16T11:44:32.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6489999999999997e-06 2024-09-16T11:44:32.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523971+0000", 2024-09-16T11:44:32.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:32.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567291+0000", 2024-09-16T11:44:32.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043319931999999998 2024-09-16T11:44:32.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567360+0000", 2024-09-16T11:44:32.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8628999999999996e-05 2024-09-16T11:44:32.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567451+0000", 2024-09-16T11:44:32.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1012000000000002e-05 2024-09-16T11:44:32.261 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.262 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.262 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.264 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19547 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4841472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.264 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:32.264 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.725819913999999, 2024-09-16T11:44:32.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043658858000000002, 2024-09-16T11:44:32.264 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.264 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.264 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19547 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:32.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525102+0000", 2024-09-16T11:44:32.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011257730000000001 2024-09-16T11:44:32.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525106+0000", 2024-09-16T11:44:32.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9910000000000004e-06 2024-09-16T11:44:32.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525116+0000", 2024-09-16T11:44:32.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0231000000000001e-05 2024-09-16T11:44:32.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525137+0000", 2024-09-16T11:44:32.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0828000000000001e-05 2024-09-16T11:44:32.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567478+0000", 2024-09-16T11:44:32.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.042340467999999999 2024-09-16T11:44:32.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567554+0000", 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6074000000000004e-05 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567635+0000", 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1493000000000001e-05 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.269 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19553 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4947968~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.269 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:32.269 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.724592852000001, 2024-09-16T11:44:32.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043450397000000002, 2024-09-16T11:44:32.269 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.269 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.269 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.269 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.269 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.269 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19553 2024-09-16T11:44:32.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.269 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:32.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:32.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:32.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1799999999999999e-07 2024-09-16T11:44:32.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525205+0000", 2024-09-16T11:44:32.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.062e-06 2024-09-16T11:44:32.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525208+0000", 2024-09-16T11:44:32.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8739999999999999e-06 2024-09-16T11:44:32.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525209+0000", 2024-09-16T11:44:32.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.725e-06 2024-09-16T11:44:32.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568520+0000", 2024-09-16T11:44:32.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043310226 2024-09-16T11:44:32.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568569+0000", 2024-09-16T11:44:32.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8959999999999999e-05 2024-09-16T11:44:32.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568654+0000", 2024-09-16T11:44:32.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5332000000000002e-05 2024-09-16T11:44:32.273 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.273 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.273 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.273 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19554 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5267456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.274 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:32.274 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.724583860000003, 2024-09-16T11:44:32.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043659587, 2024-09-16T11:44:32.274 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.274 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.274 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.274 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.274 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.274 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19554 2024-09-16T11:44:32.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.274 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:32.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:32.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527103+0000", 2024-09-16T11:44:32.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018907419999999999 2024-09-16T11:44:32.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527108+0000", 2024-09-16T11:44:32.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6040000000000003e-06 2024-09-16T11:44:32.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527118+0000", 2024-09-16T11:44:32.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:32.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527122+0000", 2024-09-16T11:44:32.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9559999999999996e-06 2024-09-16T11:44:32.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568688+0000", 2024-09-16T11:44:32.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041565116999999999 2024-09-16T11:44:32.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568768+0000", 2024-09-16T11:44:32.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0474000000000002e-05 2024-09-16T11:44:32.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568872+0000", 2024-09-16T11:44:32.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104028 2024-09-16T11:44:32.278 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.278 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.278 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.278 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19565 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6332416~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.278 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:32.278 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.722505288000001, 2024-09-16T11:44:32.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043494890000000001, 2024-09-16T11:44:32.278 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19565 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:32.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527292+0000", 2024-09-16T11:44:32.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.27e-07 2024-09-16T11:44:32.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527293+0000", 2024-09-16T11:44:32.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.917e-06 2024-09-16T11:44:32.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527299+0000", 2024-09-16T11:44:32.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6699999999999999e-06 2024-09-16T11:44:32.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527304+0000", 2024-09-16T11:44:32.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6160000000000004e-06 2024-09-16T11:44:32.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570683+0000", 2024-09-16T11:44:32.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043379171000000001 2024-09-16T11:44:32.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570733+0000", 2024-09-16T11:44:32.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0485000000000002e-05 2024-09-16T11:44:32.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570786+0000", 2024-09-16T11:44:32.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2604e-05 2024-09-16T11:44:32.283 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.283 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.283 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.283 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19566 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6758400~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.283 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:32.284 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.722488953999999, 2024-09-16T11:44:32.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043643183000000002, 2024-09-16T11:44:32.284 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.284 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.284 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.284 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.284 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.284 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19566 2024-09-16T11:44:32.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.284 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527308+0000", 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527310+0000", 2024-09-16T11:44:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.68e-06 2024-09-16T11:44:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527316+0000", 2024-09-16T11:44:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9730000000000002e-06 2024-09-16T11:44:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527319+0000", 2024-09-16T11:44:32.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.027e-06 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570810+0000", 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043491525000000003 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570882+0000", 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570951+0000", 2024-09-16T11:44:32.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8782000000000007e-05 2024-09-16T11:44:32.288 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.288 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.288 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.288 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19567 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.288 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:32.288 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.722474247999997, 2024-09-16T11:44:32.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043786291999999997, 2024-09-16T11:44:32.288 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.288 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.288 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19567 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:32.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529108+0000", 2024-09-16T11:44:32.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001785656 2024-09-16T11:44:32.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529113+0000", 2024-09-16T11:44:32.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1080000000000001e-06 2024-09-16T11:44:32.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529122+0000", 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0550000000000005e-06 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529127+0000", 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9849999999999997e-06 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570997+0000", 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041869811 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571063+0000", 2024-09-16T11:44:32.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6340999999999994e-05 2024-09-16T11:44:32.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571108+0000", 2024-09-16T11:44:32.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5336000000000001e-05 2024-09-16T11:44:32.292 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.292 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.292 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19580 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 8036352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.720498947000003, 2024-09-16T11:44:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043451596000000002, 2024-09-16T11:44:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19580 2024-09-16T11:44:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:32.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:32.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531106+0000", 2024-09-16T11:44:32.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001808179 2024-09-16T11:44:32.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531110+0000", 2024-09-16T11:44:32.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5990000000000004e-06 2024-09-16T11:44:32.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531123+0000", 2024-09-16T11:44:32.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2687000000000001e-05 2024-09-16T11:44:32.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531129+0000", 2024-09-16T11:44:32.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:44:32.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572642+0000", 2024-09-16T11:44:32.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041513474000000002 2024-09-16T11:44:32.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572691+0000", 2024-09-16T11:44:32.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8989000000000001e-05 2024-09-16T11:44:32.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572749+0000", 2024-09-16T11:44:32.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7806999999999997e-05 2024-09-16T11:44:32.297 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.297 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.297 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.297 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19709 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4849664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.297 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:32.297 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.718438906000003, 2024-09-16T11:44:32.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043537291999999998, 2024-09-16T11:44:32.298 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.298 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.298 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.298 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.298 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.298 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19709 2024-09-16T11:44:32.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.298 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.298 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:32.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:32.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531358+0000", 2024-09-16T11:44:32.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:32.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531360+0000", 2024-09-16T11:44:32.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:44:32.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531365+0000", 2024-09-16T11:44:32.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.716e-06 2024-09-16T11:44:32.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531369+0000", 2024-09-16T11:44:32.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5410000000000001e-06 2024-09-16T11:44:32.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574763+0000", 2024-09-16T11:44:32.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394359 2024-09-16T11:44:32.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574831+0000", 2024-09-16T11:44:32.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7997999999999995e-05 2024-09-16T11:44:32.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574895+0000", 2024-09-16T11:44:32.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3427000000000001e-05 2024-09-16T11:44:32.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.302 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.302 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19710 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4956160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.302 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:32.302 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.718423627, 2024-09-16T11:44:32.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043674198999999997, 2024-09-16T11:44:32.302 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.302 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.303 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19710 2024-09-16T11:44:32.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.303 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:32.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:32.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533105+0000", 2024-09-16T11:44:32.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001731961 2024-09-16T11:44:32.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533109+0000", 2024-09-16T11:44:32.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3080000000000001e-06 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533120+0000", 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0625000000000001e-05 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533125+0000", 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574923+0000", 2024-09-16T11:44:32.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041797187 2024-09-16T11:44:32.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574991+0000", 2024-09-16T11:44:32.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8752999999999997e-05 2024-09-16T11:44:32.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.575047+0000", 2024-09-16T11:44:32.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5596000000000001e-05 2024-09-16T11:44:32.307 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.307 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.307 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.307 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19719 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5488640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.307 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:32.307 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.716451878000001, 2024-09-16T11:44:32.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043412013999999999, 2024-09-16T11:44:32.307 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.307 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.308 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.308 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.308 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.308 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19719 2024-09-16T11:44:32.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.308 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:32.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533345+0000", 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533347+0000", 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8139999999999999e-06 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533352+0000", 2024-09-16T11:44:32.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4970000000000001e-06 2024-09-16T11:44:32.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533355+0000", 2024-09-16T11:44:32.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.711e-06 2024-09-16T11:44:32.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576629+0000", 2024-09-16T11:44:32.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043274295999999997 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576706+0000", 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6711000000000005e-05 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576757+0000", 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0509e-05 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.312 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19720 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5808128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.312 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:32.312 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.716439229000002, 2024-09-16T11:44:32.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043577006000000001, 2024-09-16T11:44:32.312 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.312 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.312 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.312 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.312 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.312 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19720 2024-09-16T11:44:32.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.312 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:32.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:32.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:32.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.35e-07 2024-09-16T11:44:32.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533359+0000", 2024-09-16T11:44:32.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:44:32.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533364+0000", 2024-09-16T11:44:32.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4349999999999996e-06 2024-09-16T11:44:32.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533367+0000", 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2349999999999999e-06 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576786+0000", 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043418885999999997 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576862+0000", 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6172000000000002e-05 2024-09-16T11:44:32.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576934+0000", 2024-09-16T11:44:32.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1891999999999995e-05 2024-09-16T11:44:32.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.316 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.316 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19721 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6234112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.316 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:32.316 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.716425708999999, 2024-09-16T11:44:32.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043681656999999999, 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19721 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-07 2024-09-16T11:44:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533373+0000", 2024-09-16T11:44:32.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.973e-06 2024-09-16T11:44:32.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533377+0000", 2024-09-16T11:44:32.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.403e-06 2024-09-16T11:44:32.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533379+0000", 2024-09-16T11:44:32.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.058e-06 2024-09-16T11:44:32.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576959+0000", 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043579433000000001 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577015+0000", 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5859000000000002e-05 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577052+0000", 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7620999999999998e-05 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.320 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.321 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.321 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19722 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6340608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.321 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:32.321 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.716414982000003, 2024-09-16T11:44:32.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043854922999999997, 2024-09-16T11:44:32.321 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.321 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.321 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.321 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.321 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.321 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19722 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535103+0000", 2024-09-16T11:44:32.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017217020000000001 2024-09-16T11:44:32.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535107+0000", 2024-09-16T11:44:32.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2819999999999998e-06 2024-09-16T11:44:32.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535118+0000", 2024-09-16T11:44:32.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0148e-05 2024-09-16T11:44:32.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535123+0000", 2024-09-16T11:44:32.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:44:32.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577091+0000", 2024-09-16T11:44:32.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041967930000000001 2024-09-16T11:44:32.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577163+0000", 2024-09-16T11:44:32.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1853999999999993e-05 2024-09-16T11:44:32.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577236+0000", 2024-09-16T11:44:32.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3699e-05 2024-09-16T11:44:32.325 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.325 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.325 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.325 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19732 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6447104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.325 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.714555126, 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043399048000000003, 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19732 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:32.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535242+0000", 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535243+0000", 2024-09-16T11:44:32.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.793e-06 2024-09-16T11:44:32.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535249+0000", 2024-09-16T11:44:32.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.643e-06 2024-09-16T11:44:32.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535252+0000", 2024-09-16T11:44:32.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:44:32.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578501+0000", 2024-09-16T11:44:32.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043248343000000002 2024-09-16T11:44:32.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578576+0000", 2024-09-16T11:44:32.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5544999999999996e-05 2024-09-16T11:44:32.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578640+0000", 2024-09-16T11:44:32.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4018999999999998e-05 2024-09-16T11:44:32.330 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.330 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.330 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19733 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6873088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.714534284999999, 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043549681, 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19733 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:32.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:32.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535263+0000", 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535265+0000", 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9369999999999998e-06 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535271+0000", 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9719999999999997e-06 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535274+0000", 2024-09-16T11:44:32.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:44:32.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578668+0000", 2024-09-16T11:44:32.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043393355000000002 2024-09-16T11:44:32.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578744+0000", 2024-09-16T11:44:32.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6853000000000004e-05 2024-09-16T11:44:32.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578812+0000", 2024-09-16T11:44:32.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.737e-05 2024-09-16T11:44:32.335 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.335 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.335 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.335 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19734 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7299072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.335 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:32.335 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.714518538, 2024-09-16T11:44:32.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043699189999999999, 2024-09-16T11:44:32.336 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.336 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.336 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.336 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.336 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.336 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19734 2024-09-16T11:44:32.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.336 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:32.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:32.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537101+0000", 2024-09-16T11:44:32.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001823098 2024-09-16T11:44:32.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537105+0000", 2024-09-16T11:44:32.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:32.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537119+0000", 2024-09-16T11:44:32.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614999999999999e-05 2024-09-16T11:44:32.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537125+0000", 2024-09-16T11:44:32.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:44:32.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578838+0000", 2024-09-16T11:44:32.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041712605999999999 2024-09-16T11:44:32.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578912+0000", 2024-09-16T11:44:32.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4396999999999999e-05 2024-09-16T11:44:32.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578977+0000", 2024-09-16T11:44:32.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4858999999999997e-05 2024-09-16T11:44:32.340 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.340 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.340 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.340 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19851 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4431872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.340 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:32.340 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 36.712437794000003, 2024-09-16T11:44:32.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043564540999999998, 2024-09-16T11:44:32.340 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.340 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:32.340 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.340 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.341 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.341 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19851 2024-09-16T11:44:32.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.341 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:32.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:32.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539100+0000", 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017412079999999999 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539105+0000", 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539116+0000", 2024-09-16T11:44:32.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1824999999999999e-05 2024-09-16T11:44:32.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539122+0000", 2024-09-16T11:44:32.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9859999999999999e-06 2024-09-16T11:44:32.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580764+0000", 2024-09-16T11:44:32.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041641299 2024-09-16T11:44:32.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580839+0000", 2024-09-16T11:44:32.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5167999999999996e-05 2024-09-16T11:44:32.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580923+0000", 2024-09-16T11:44:32.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4314999999999995e-05 2024-09-16T11:44:32.344 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.344 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.344 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.345 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55904 4.14 4:2d2dfe38:::rbd_data.11c1b01b799a.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2068993~28159] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T11:44:32.345 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:32.345 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.1580608940000001, 2024-09-16T11:44:32.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.90010917700000004, 2024-09-16T11:44:32.345 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:32.345 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:32.345 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:32.345 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:32.345 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.345 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55904 2024-09-16T11:44:32.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.345 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:32.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:32.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:32.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:32.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:32.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:32.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:32.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091737+0000", 2024-09-16T11:44:32.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.254e-06 2024-09-16T11:44:32.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:32.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091759+0000", 2024-09-16T11:44:32.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2487999999999999e-05 2024-09-16T11:44:32.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:32.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091769+0000", 2024-09-16T11:44:32.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0199e-05 2024-09-16T11:44:32.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091774+0000", 2024-09-16T11:44:32.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:44:32.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091830+0000", 2024-09-16T11:44:32.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6700999999999998e-05 2024-09-16T11:44:32.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:44:32.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091840+0000", 2024-09-16T11:44:32.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0128e-05 2024-09-16T11:44:32.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990355+0000", 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.89851413099999999 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990509+0000", 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000154442 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990626+0000", 2024-09-16T11:44:32.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000116928 2024-09-16T11:44:32.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:32.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991526+0000", 2024-09-16T11:44:32.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00089961800000000003 2024-09-16T11:44:32.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:32.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991793+0000", 2024-09-16T11:44:32.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026779000000000001 2024-09-16T11:44:32.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991824+0000", 2024-09-16T11:44:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0380999999999999e-05 2024-09-16T11:44:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991845+0000", 2024-09-16T11:44:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0919000000000001e-05 2024-09-16T11:44:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:32.351 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:32.352 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.352 INFO:teuthology.orchestra.run.smithi070.stdout:5833 punch from 0x4ca06d9 to 0x4cab9cf, (0xb2f6 bytes) 2024-09-16T11:44:32.353 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:32.355 INFO:teuthology.orchestra.run.smithi070.stdout:5834 trunc from 0xc123fae to 0x65f2cb9 2024-09-16T11:44:32.355 INFO:teuthology.orchestra.run.smithi070.stdout:5835 writesame 0x1e02460 thru 0x1e0bd45 data_size 0x1(0x98e6 bytes) 2024-09-16T11:44:32.355 INFO:teuthology.orchestra.run.smithi070.stdout:5836 flatten 2024-09-16T11:44:32.355 INFO:teuthology.orchestra.run.smithi070.stdout:5837 read 0x3bceaab thru 0x3bdb890 (0xcde6 bytes) 2024-09-16T11:44:32.356 INFO:teuthology.orchestra.run.smithi070.stdout:5838 write 0xe45253f thru 0xe45e25d (0xbd1f bytes) 2024-09-16T11:44:32.356 INFO:teuthology.orchestra.run.smithi070.stdout:5839 punch from 0x8436cb9 to 0x8440b4f, (0x9e96 bytes) 2024-09-16T11:44:32.356 INFO:teuthology.orchestra.run.smithi070.stdout:5841 punch from 0x1a71c8f to 0x1a7f4ee, (0xd85f bytes) 2024-09-16T11:44:32.356 INFO:teuthology.orchestra.run.smithi070.stdout:5843 read 0x448c344 thru 0x449096a (0x4627 bytes) 2024-09-16T11:44:32.356 INFO:teuthology.orchestra.run.smithi070.stdout:5845 read 0xd100fb8 thru 0xd10a8ef (0x9938 bytes) 2024-09-16T11:44:32.356 INFO:teuthology.orchestra.run.smithi070.stdout:5846 writesame 0x13e6c3f thru 0x13f0d08 data_size 0xb(0xa0ca bytes) 2024-09-16T11:44:32.356 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.356 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.356 INFO:teuthology.orchestra.run.smithi070.stdout:5849 read 0xc60741a thru 0xc609fc2 (0x2ba9 bytes) 2024-09-16T11:44:32.356 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.356 INFO:teuthology.orchestra.run.smithi070.stdout:5851 read 0x7eb595b thru 0x7ebdb5f (0x8205 bytes) 2024-09-16T11:44:32.356 INFO:teuthology.orchestra.run.smithi070.stdout:5853 trunc from 0xe45e25e to 0xecd97ca 2024-09-16T11:44:32.357 INFO:teuthology.orchestra.run.smithi070.stdout:5855 write 0x6d34b43 thru 0x6d4120b (0xc6c9 bytes) 2024-09-16T11:44:32.357 INFO:teuthology.orchestra.run.smithi070.stdout:5856 writesame 0x95df1c2 thru 0x95eeabd data_size 0xa(0xf8fc bytes) 2024-09-16T11:44:32.357 INFO:teuthology.orchestra.run.smithi070.stdout:5857 read 0xe10ddf2 thru 0xe11106c (0x327b bytes) 2024-09-16T11:44:32.357 INFO:teuthology.orchestra.run.smithi070.stdout:5858 trunc from 0xecd97ca to 0xd14fb8c 2024-09-16T11:44:32.357 INFO:teuthology.orchestra.run.smithi070.stdout:5859 trunc from 0xd14fb8c to 0xac6c959 2024-09-16T11:44:32.357 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:32.351+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:32.362 INFO:teuthology.orchestra.run.smithi070.stdout:5860 write 0xb33eb3f thru 0xb34c19e (0xd660 bytes) 2024-09-16T11:44:32.365 INFO:teuthology.orchestra.run.smithi070.stdout:5862 read 0x9c0527 thru 0x9ce297 (0xdd71 bytes) 2024-09-16T11:44:32.365 INFO:teuthology.orchestra.run.smithi070.stdout:5863 punch from 0xa412bfe to 0xa41adad, (0x81af bytes) 2024-09-16T11:44:32.366 INFO:teuthology.orchestra.run.smithi070.stdout:5864 read 0x2ee6117 thru 0x2eec1ac (0x6096 bytes) 2024-09-16T11:44:32.372 INFO:teuthology.orchestra.run.smithi070.stdout:5866 write 0xc6dc086 thru 0xc6e1b50 (0x5acb bytes) 2024-09-16T11:44:32.376 INFO:teuthology.orchestra.run.smithi070.stdout:5867 read 0xab6e6cd thru 0xab7a9b6 (0xc2ea bytes) 2024-09-16T11:44:32.376 INFO:teuthology.orchestra.run.smithi070.stdout:5869 read 0x84cb709 thru 0x84d2fb5 (0x78ad bytes) 2024-09-16T11:44:32.376 INFO:teuthology.orchestra.run.smithi070.stdout:5870 read 0x8207b6a thru 0x820d1d6 (0x566d bytes) 2024-09-16T11:44:32.376 INFO:teuthology.orchestra.run.smithi070.stdout:5871 writesame 0x5f7ced0 thru 0x5f835c0 data_size 0x13(0x66f1 bytes) 2024-09-16T11:44:32.378 INFO:teuthology.orchestra.run.smithi070.stdout:5873 write 0x5695be5 thru 0x56a4a63 (0xee7f bytes) 2024-09-16T11:44:32.380 INFO:teuthology.orchestra.run.smithi070.stdout:5874 write 0x9333f36 thru 0x933c1c4 (0x828f bytes) 2024-09-16T11:44:32.383 INFO:teuthology.orchestra.run.smithi070.stdout:5875 writesame 0xad47af3 thru 0xad4bd3b data_size 0x1(0x4249 bytes) 2024-09-16T11:44:32.392 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.397 INFO:teuthology.orchestra.run.smithi070.stdout:5877 trunc from 0xc6e1b51 to 0x8a9d149 2024-09-16T11:44:32.403 INFO:teuthology.orchestra.run.smithi070.stdout:5880 read 0x338f563 thru 0x339aa52 (0xb4f0 bytes) 2024-09-16T11:44:32.404 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.404 INFO:teuthology.orchestra.run.smithi070.stdout:5886 punch from 0x4c535e0 to 0x4c5bb2e, (0x854e bytes) 2024-09-16T11:44:32.404 INFO:teuthology.orchestra.run.smithi070.stdout:5887 read 0x411f884 thru 0x4121570 (0x1ced bytes) 2024-09-16T11:44:32.404 INFO:teuthology.orchestra.run.smithi070.stdout:5888 read 0x1e63487 thru 0x1e69280 (0x5dfa bytes) 2024-09-16T11:44:32.405 INFO:teuthology.orchestra.run.smithi070.stdout:5889 punch from 0x1db821e to 0x1dbe1c6, (0x5fa8 bytes) 2024-09-16T11:44:32.415 INFO:teuthology.orchestra.run.smithi070.stdout:5890 write 0xcff2d90 thru 0xcffa926 (0x7b97 bytes) 2024-09-16T11:44:32.416 INFO:teuthology.orchestra.run.smithi070.stdout:5891 punch from 0x150a02b to 0x1517bed, (0xdbc2 bytes) 2024-09-16T11:44:32.416 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.416 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.417 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.417 INFO:teuthology.orchestra.run.smithi070.stdout:5898 read 0xfbd6c7 thru 0xfc8ba3 (0xb4dd bytes) 2024-09-16T11:44:32.417 INFO:teuthology.orchestra.run.smithi070.stdout:5899 read 0x1071f20 thru 0x1074e45 (0x2f26 bytes) 2024-09-16T11:44:32.417 INFO:teuthology.orchestra.run.smithi070.stdout:5901 punch from 0x9bff3fd to 0x9c0b068, (0xbc6b bytes) 2024-09-16T11:44:32.420 INFO:teuthology.orchestra.run.smithi070.stdout:5902 trunc from 0xcffa927 to 0xa30144c 2024-09-16T11:44:32.427 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.427 INFO:teuthology.orchestra.run.smithi070.stdout:5907 read 0x822a2d8 thru 0x82322a4 (0x7fcd bytes) 2024-09-16T11:44:32.427 INFO:teuthology.orchestra.run.smithi070.stdout:5909 writesame 0x285aae3 thru 0x2867abd data_size 0x1(0xcfdb bytes) 2024-09-16T11:44:32.452 INFO:teuthology.orchestra.run.smithi070.stdout:5911 writesame 0xe03a2a thru 0xe07ff8 data_size 0x25(0x45cf bytes) 2024-09-16T11:44:32.453 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:32.451+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:32.453 INFO:teuthology.orchestra.run.smithi070.stdout:5912 read 0x6d655a8 thru 0x6d74988 (0xf3e1 bytes) 2024-09-16T11:44:32.454 INFO:teuthology.orchestra.run.smithi070.stdout:5913 read 0x55133b4 thru 0x5519bdc (0x6829 bytes) 2024-09-16T11:44:32.454 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:32.454 INFO:teuthology.orchestra.run.smithi070.stdout:5916 clone 39 order 20 su 8192 sc 11 2024-09-16T11:44:32.458 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:32.458 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:32.458 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:32.458 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:32.469 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:32.553 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:32.552+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:32.567 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:32.567 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:32.567 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:32.568 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:32.568 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:32.576 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:32.655 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:32.654+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:32.716 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:32.716 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:32.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:32.716 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:32.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.717 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.717 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:32.717 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351976815, 2024-09-16T11:44:32.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:44:32.717 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.717 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.718 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.718 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:44:32.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.718 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:32.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:32.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:32.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:44:32.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:44:32.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:44:32.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:44:32.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:44:32.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:44:32.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:44:32.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:44:32.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:44:32.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:44:32.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:44:32.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:44:32.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:44:32.721 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.721 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.722 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.722 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.722 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:32.722 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351954786, 2024-09-16T11:44:32.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:44:32.722 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.722 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.722 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:32.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:32.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:44:32.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:44:32.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:44:32.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:44:32.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:44:32.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:44:32.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:44:32.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.727 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.727 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:32.727 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351940020999997, 2024-09-16T11:44:32.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:44:32.727 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.727 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.727 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:44:32.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.728 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:32.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:32.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:32.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:44:32.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:44:32.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:44:32.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:44:32.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:44:32.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:44:32.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:44:32.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:44:32.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:44:32.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:44:32.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:44:32.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:44:32.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:44:32.732 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.732 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.733 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.733 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.733 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:32.733 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351919952999999, 2024-09-16T11:44:32.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:44:32.733 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.733 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.734 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:44:32.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.734 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:32.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:32.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:44:32.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:44:32.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:44:32.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:44:32.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:44:32.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:44:32.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:44:32.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:44:32.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:44:32.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:44:32.737 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.737 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.737 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.738 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.738 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:32.738 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351889116999999, 2024-09-16T11:44:32.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:44:32.738 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.738 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.738 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:44:32.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.739 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:32.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:32.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:32.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:44:32.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:44:32.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:44:32.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:44:32.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:44:32.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:44:32.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:44:32.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:44:32.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:44:32.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:44:32.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:44:32.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:44:32.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:44:32.744 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.744 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.744 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.744 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.744 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:32.744 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351869596999997, 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:44:32.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:44:32.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:44:32.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:44:32.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:44:32.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:44:32.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.748 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351842017999999, 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:44:32.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.750 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:32.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:32.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:44:32.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:44:32.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:44:32.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:44:32.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:44:32.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351823322999998, 2024-09-16T11:44:32.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:44:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:44:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:44:32.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:44:32.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:44:32.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:44:32.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:44:32.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:44:32.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:44:32.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:44:32.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:44:32.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:44:32.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:44:32.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:44:32.758 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.758 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.758 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.759 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.759 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:32.759 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351808314000003, 2024-09-16T11:44:32.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:44:32.759 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.759 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.759 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.760 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:44:32.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.760 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:32.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:32.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:44:32.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:32.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:44:32.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:44:32.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:44:32.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:44:32.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:44:32.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:44:32.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:44:32.764 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.764 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.764 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.764 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.764 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:32.764 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351776215999998, 2024-09-16T11:44:32.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:44:32.764 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.764 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.765 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.765 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.765 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:44:32.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.765 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:32.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:32.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:44:32.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:44:32.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:44:32.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:44:32.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:44:32.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:44:32.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:44:32.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:44:32.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:44:32.768 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.768 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.768 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.769 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.769 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:32.769 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351752597000001, 2024-09-16T11:44:32.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:44:32.769 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.769 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.769 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.769 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.769 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.769 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:44:32.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.770 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:32.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:32.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:44:32.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:44:32.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:44:32.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:44:32.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:44:32.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351737868000001, 2024-09-16T11:44:32.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:44:32.774 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.774 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.774 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:44:32.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.774 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:32.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:44:32.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:44:32.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:44:32.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:44:32.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:44:32.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:44:32.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:44:32.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:44:32.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:44:32.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:44:32.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:44:32.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:44:32.777 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351716044, 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.778 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:44:32.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:32.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:44:32.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:44:32.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:44:32.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:44:32.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:44:32.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:44:32.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.784 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.784 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.784 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:32.784 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351701394000003, 2024-09-16T11:44:32.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:44:32.784 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.784 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.784 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.785 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:44:32.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.785 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:32.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:32.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:44:32.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:44:32.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:44:32.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:44:32.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:44:32.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:44:32.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:44:32.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:44:32.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:44:32.788 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.788 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.788 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351679820999998, 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:32.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:32.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:32.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:44:32.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:44:32.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:44:32.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:44:32.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:44:32.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:44:32.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:32.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:44:32.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:44:32.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:44:32.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:44:32.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:44:32.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:44:32.793 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.793 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.793 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.793 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.793 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:32.793 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351660375000002, 2024-09-16T11:44:32.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:32.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:44:32.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:32.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:44:32.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:44:32.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:44:32.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:44:32.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:44:32.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:44:32.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:44:32.797 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.797 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351623678000003, 2024-09-16T11:44:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:44:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:44:32.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.799 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:32.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:32.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:44:32.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:44:32.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:44:32.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:44:32.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:44:32.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:32.802 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351601731999999, 2024-09-16T11:44:32.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:44:32.803 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.803 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.803 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.803 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.803 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.803 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:44:32.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.803 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:32.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:44:32.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:32.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:44:32.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:44:32.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:44:32.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:44:32.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:44:32.806 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.807 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.807 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.807 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:32.807 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351579846, 2024-09-16T11:44:32.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:44:32.807 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.807 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.808 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:44:32.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.808 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:32.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:32.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:32.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:44:32.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:44:32.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:44:32.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:44:32.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:44:32.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:44:32.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:44:32.811 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.811 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.811 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.811 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:32.812 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:32.812 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.351565127000001, 2024-09-16T11:44:32.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:44:32.812 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:32.812 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:32.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:32.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:32.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:32.812 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:44:32.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.812 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:32.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:32.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:32.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:32.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:32.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:32.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:32.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:32.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:32.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:44:32.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:44:32.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:32.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:44:32.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:44:32.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:32.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:44:32.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:44:32.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:32.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:32.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:32.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:44:32.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:44:32.816 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.816 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.816 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.816 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:32.816 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:32.816 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:32.817 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:32.817 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:32.780+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:32.883 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:32.881+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:32.922 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:32.922 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:32.922 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:32.922 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:32.931 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:32.984 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:32.983+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:33.010 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:44:33.029 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:33.029 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:33.030 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:33.030 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:33.039 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:33.085 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:33.083+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:33.139 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19943 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2416640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:33.139 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609969282999998, 2024-09-16T11:44:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027947257999999999, 2024-09-16T11:44:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.140 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19943 2024-09-16T11:44:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5999999999999999e-07 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526297+0000", 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8059999999999999e-06 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526303+0000", 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6579999999999999e-06 2024-09-16T11:44:33.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526306+0000", 2024-09-16T11:44:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3969999999999998e-06 2024-09-16T11:44:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554165+0000", 2024-09-16T11:44:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027858519000000002 2024-09-16T11:44:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554201+0000", 2024-09-16T11:44:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5908000000000002e-05 2024-09-16T11:44:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554242+0000", 2024-09-16T11:44:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1510000000000001e-05 2024-09-16T11:44:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19944 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2842624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609954387999998, 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028052639000000001, 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19944 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:44:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526312+0000", 2024-09-16T11:44:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:44:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526317+0000", 2024-09-16T11:44:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5910000000000003e-06 2024-09-16T11:44:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526321+0000", 2024-09-16T11:44:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2339999999999999e-06 2024-09-16T11:44:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554263+0000", 2024-09-16T11:44:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027942181 2024-09-16T11:44:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554307+0000", 2024-09-16T11:44:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4499999999999997e-05 2024-09-16T11:44:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554362+0000", 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5078999999999999e-05 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19945 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3162112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609939832999999, 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028145673999999999, 2024-09-16T11:44:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19945 2024-09-16T11:44:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526325+0000", 2024-09-16T11:44:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:44:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526326+0000", 2024-09-16T11:44:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.733e-06 2024-09-16T11:44:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526332+0000", 2024-09-16T11:44:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:44:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526336+0000", 2024-09-16T11:44:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8410000000000002e-06 2024-09-16T11:44:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554389+0000", 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028053576 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554427+0000", 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7299999999999999e-05 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554470+0000", 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3195000000000002e-05 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19946 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3268608~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609924253999999, 2024-09-16T11:44:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028232587, 2024-09-16T11:44:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19946 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526346+0000", 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6459999999999998e-06 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526352+0000", 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1070000000000001e-06 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526355+0000", 2024-09-16T11:44:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:44:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554493+0000", 2024-09-16T11:44:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.02813746 2024-09-16T11:44:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554530+0000", 2024-09-16T11:44:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6881999999999997e-05 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554572+0000", 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2688999999999997e-05 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19947 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4014080~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609904899999997, 2024-09-16T11:44:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028315646999999999, 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19947 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526360+0000", 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.01e-07 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526361+0000", 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8300000000000001e-06 2024-09-16T11:44:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526369+0000", 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526377+0000", 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0909999999999993e-06 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554593+0000", 2024-09-16T11:44:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028216657999999999 2024-09-16T11:44:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554632+0000", 2024-09-16T11:44:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8828000000000002e-05 2024-09-16T11:44:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554675+0000", 2024-09-16T11:44:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2710000000000003e-05 2024-09-16T11:44:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19948 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3588096~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609882501000001, 2024-09-16T11:44:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028394545, 2024-09-16T11:44:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19948 2024-09-16T11:44:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9200000000000001e-07 2024-09-16T11:44:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526384+0000", 2024-09-16T11:44:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:44:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526391+0000", 2024-09-16T11:44:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5150000000000003e-06 2024-09-16T11:44:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526394+0000", 2024-09-16T11:44:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4240000000000002e-06 2024-09-16T11:44:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554695+0000", 2024-09-16T11:44:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028301218999999999 2024-09-16T11:44:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554731+0000", 2024-09-16T11:44:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6298999999999999e-05 2024-09-16T11:44:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554776+0000", 2024-09-16T11:44:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4675999999999997e-05 2024-09-16T11:44:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19949 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1351680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609866228999998, 2024-09-16T11:44:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028490081, 2024-09-16T11:44:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19949 2024-09-16T11:44:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:44:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526400+0000", 2024-09-16T11:44:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7099999999999999e-06 2024-09-16T11:44:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526406+0000", 2024-09-16T11:44:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5140000000000001e-06 2024-09-16T11:44:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526409+0000", 2024-09-16T11:44:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5360000000000001e-06 2024-09-16T11:44:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554801+0000", 2024-09-16T11:44:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028392378999999999 2024-09-16T11:44:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554843+0000", 2024-09-16T11:44:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1056000000000001e-05 2024-09-16T11:44:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554888+0000", 2024-09-16T11:44:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466999999999997e-05 2024-09-16T11:44:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19950 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1777664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609851233000001, 2024-09-16T11:44:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028661844999999998, 2024-09-16T11:44:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19950 2024-09-16T11:44:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526415+0000", 2024-09-16T11:44:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.776e-06 2024-09-16T11:44:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526422+0000", 2024-09-16T11:44:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6930000000000001e-06 2024-09-16T11:44:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526425+0000", 2024-09-16T11:44:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554923+0000", 2024-09-16T11:44:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028498102000000001 2024-09-16T11:44:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555000+0000", 2024-09-16T11:44:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6619999999999995e-05 2024-09-16T11:44:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555075+0000", 2024-09-16T11:44:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4931999999999993e-05 2024-09-16T11:44:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19951 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2097152~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609835422000003, 2024-09-16T11:44:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028755718, 2024-09-16T11:44:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19951 2024-09-16T11:44:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526431+0000", 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526441+0000", 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7790000000000007e-06 2024-09-16T11:44:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526445+0000", 2024-09-16T11:44:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.634e-06 2024-09-16T11:44:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555100+0000", 2024-09-16T11:44:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654722000000001 2024-09-16T11:44:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555138+0000", 2024-09-16T11:44:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7996e-05 2024-09-16T11:44:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555184+0000", 2024-09-16T11:44:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6427999999999997e-05 2024-09-16T11:44:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19952 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2203648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609814862, 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028901157, 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19952 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526450+0000", 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526452+0000", 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9039999999999999e-06 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526465+0000", 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3559e-05 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526470+0000", 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555217+0000", 2024-09-16T11:44:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028747287999999999 2024-09-16T11:44:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555272+0000", 2024-09-16T11:44:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4885000000000001e-05 2024-09-16T11:44:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555350+0000", 2024-09-16T11:44:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8428000000000002e-05 2024-09-16T11:44:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19953 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2523136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609790269999998, 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029004309999999998, 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19953 2024-09-16T11:44:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-16T11:44:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526476+0000", 2024-09-16T11:44:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9810000000000002e-06 2024-09-16T11:44:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526482+0000", 2024-09-16T11:44:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526485+0000", 2024-09-16T11:44:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2760000000000001e-06 2024-09-16T11:44:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555383+0000", 2024-09-16T11:44:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028898113 2024-09-16T11:44:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555433+0000", 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9753999999999998e-05 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555478+0000", 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5006999999999998e-05 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19954 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2949120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609775272999997, 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029165054999999999, 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19954 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1799999999999995e-07 2024-09-16T11:44:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526491+0000", 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7320000000000001e-06 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526497+0000", 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6869999999999999e-06 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526500+0000", 2024-09-16T11:44:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:44:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555513+0000", 2024-09-16T11:44:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029012626 2024-09-16T11:44:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555576+0000", 2024-09-16T11:44:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3288999999999997e-05 2024-09-16T11:44:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555654+0000", 2024-09-16T11:44:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7861999999999994e-05 2024-09-16T11:44:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19955 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3375104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609755970999998, 2024-09-16T11:44:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029241979000000001, 2024-09-16T11:44:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19955 2024-09-16T11:44:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526509+0000", 2024-09-16T11:44:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-09-16T11:44:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526510+0000", 2024-09-16T11:44:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.844e-06 2024-09-16T11:44:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526516+0000", 2024-09-16T11:44:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9039999999999997e-06 2024-09-16T11:44:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526520+0000", 2024-09-16T11:44:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3639999999999999e-06 2024-09-16T11:44:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555669+0000", 2024-09-16T11:44:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029149628 2024-09-16T11:44:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555706+0000", 2024-09-16T11:44:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7017000000000002e-05 2024-09-16T11:44:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555750+0000", 2024-09-16T11:44:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3856999999999997e-05 2024-09-16T11:44:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19956 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3694592~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609741016000001, 2024-09-16T11:44:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029398799999999999, 2024-09-16T11:44:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19956 2024-09-16T11:44:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526524+0000", 2024-09-16T11:44:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:44:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526525+0000", 2024-09-16T11:44:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-16T11:44:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526531+0000", 2024-09-16T11:44:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6010000000000002e-06 2024-09-16T11:44:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526535+0000", 2024-09-16T11:44:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.597e-06 2024-09-16T11:44:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555785+0000", 2024-09-16T11:44:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029250235999999999 2024-09-16T11:44:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555845+0000", 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0393000000000003e-05 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555922+0000", 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6710000000000002e-05 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19957 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4120576~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.609725863000001, 2024-09-16T11:44:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029481157000000001, 2024-09-16T11:44:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19957 2024-09-16T11:44:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526539+0000", 2024-09-16T11:44:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526540+0000", 2024-09-16T11:44:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.759e-06 2024-09-16T11:44:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526546+0000", 2024-09-16T11:44:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1079999999999998e-06 2024-09-16T11:44:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526550+0000", 2024-09-16T11:44:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.269e-06 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555939+0000", 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029389444000000001 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555977+0000", 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7487999999999998e-05 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.556019+0000", 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2756999999999997e-05 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:28959 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.126869450999997, 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031145285000000002, 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 28959 2024-09-16T11:44:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009396+0000", 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.296e-06 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030864+0000", 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.021468451 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030892+0000", 2024-09-16T11:44:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7555999999999998e-05 2024-09-16T11:44:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030902+0000", 2024-09-16T11:44:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9550000000000004e-06 2024-09-16T11:44:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030979+0000", 2024-09-16T11:44:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7137000000000004e-05 2024-09-16T11:44:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031051+0000", 2024-09-16T11:44:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1475000000000003e-05 2024-09-16T11:44:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031389+0000", 2024-09-16T11:44:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033827100000000001 2024-09-16T11:44:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.039264+0000", 2024-09-16T11:44:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078751700000000008 2024-09-16T11:44:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040484+0000", 2024-09-16T11:44:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012199579999999999 2024-09-16T11:44:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040517+0000", 2024-09-16T11:44:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2960999999999998e-05 2024-09-16T11:44:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040540+0000", 2024-09-16T11:44:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3054999999999999e-05 2024-09-16T11:44:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30605 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 3915776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.892563192000001, 2024-09-16T11:44:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938655999999999, 2024-09-16T11:44:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30605 2024-09-16T11:44:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243704+0000", 2024-09-16T11:44:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2199999999999999e-06 2024-09-16T11:44:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243710+0000", 2024-09-16T11:44:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6340000000000003e-06 2024-09-16T11:44:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243714+0000", 2024-09-16T11:44:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5030000000000002e-06 2024-09-16T11:44:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271450+0000", 2024-09-16T11:44:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027736035999999999 2024-09-16T11:44:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271536+0000", 2024-09-16T11:44:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6521999999999996e-05 2024-09-16T11:44:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271640+0000", 2024-09-16T11:44:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103341 2024-09-16T11:44:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30607 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4046848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.892540037, 2024-09-16T11:44:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028130446, 2024-09-16T11:44:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30607 2024-09-16T11:44:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243725+0000", 2024-09-16T11:44:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2399999999999998e-07 2024-09-16T11:44:33.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243727+0000", 2024-09-16T11:44:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3379999999999999e-06 2024-09-16T11:44:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243733+0000", 2024-09-16T11:44:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4280000000000001e-06 2024-09-16T11:44:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243737+0000", 2024-09-16T11:44:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8990000000000003e-06 2024-09-16T11:44:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271676+0000", 2024-09-16T11:44:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938481000000001 2024-09-16T11:44:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271757+0000", 2024-09-16T11:44:33.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1396999999999993e-05 2024-09-16T11:44:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271855+0000", 2024-09-16T11:44:33.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7379000000000004e-05 2024-09-16T11:44:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.248 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30608 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.892523185999998, 2024-09-16T11:44:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028323879, 2024-09-16T11:44:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.249 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30608 2024-09-16T11:44:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:44:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243743+0000", 2024-09-16T11:44:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.762e-06 2024-09-16T11:44:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243749+0000", 2024-09-16T11:44:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5670000000000002e-06 2024-09-16T11:44:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243758+0000", 2024-09-16T11:44:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8309999999999996e-06 2024-09-16T11:44:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271879+0000", 2024-09-16T11:44:33.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028120501999999999 2024-09-16T11:44:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271965+0000", 2024-09-16T11:44:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5755999999999997e-05 2024-09-16T11:44:33.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.272065+0000", 2024-09-16T11:44:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000100101 2024-09-16T11:44:33.258 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:56045 4.15 4:ab2ce1b7:::rbd_data.11c1b01b799a.0000000000000020:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-16T11:44:33.259 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:44:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.957239935, 2024-09-16T11:44:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032248165000000002, 2024-09-16T11:44:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:33.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 56045 2024-09-16T11:44:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:44:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:44:33.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179026+0000", 2024-09-16T11:44:33.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.556e-06 2024-09-16T11:44:33.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179032+0000", 2024-09-16T11:44:33.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5379999999999999e-06 2024-09-16T11:44:33.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179045+0000", 2024-09-16T11:44:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.277e-05 2024-09-16T11:44:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179067+0000", 2024-09-16T11:44:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.243e-05 2024-09-16T11:44:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179120+0000", 2024-09-16T11:44:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2726000000000003e-05 2024-09-16T11:44:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179279+0000", 2024-09-16T11:44:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015865899999999999 2024-09-16T11:44:33.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.185163+0000", 2024-09-16T11:44:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0058837639999999997 2024-09-16T11:44:33.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211197+0000", 2024-09-16T11:44:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026034130999999999 2024-09-16T11:44:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211206+0000", 2024-09-16T11:44:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2539999999999998e-06 2024-09-16T11:44:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211245+0000", 2024-09-16T11:44:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8924999999999997e-05 2024-09-16T11:44:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211272+0000", 2024-09-16T11:44:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7412000000000001e-05 2024-09-16T11:44:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.273 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.274 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.274 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:33.276 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-16T11:44:33.276 INFO:teuthology.orchestra.run.smithi070.stdout:5918 trunc from 0xa30144c to 0xabc91bd 2024-09-16T11:44:33.276 INFO:teuthology.orchestra.run.smithi070.stdout:5920 clone 40 order 22 su 1048576 sc 9 2024-09-16T11:44:33.277 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:33.277 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:33.274+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:33.377 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:33.375+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:33.416 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:33.417 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:33.417 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:33.417 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:33.425 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:33.477 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:33.476+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:33.564 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:33.564 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:33.564 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:33.565 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:33.565 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:33.574 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:33.578 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:33.576+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:33.674 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:33.674 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:33.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:33.674 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19282 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.199015205000002, 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521151000000002, 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19282 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:33.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:33.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472838+0000", 2024-09-16T11:44:33.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472848+0000", 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9799999999999993e-06 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472858+0000", 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9429999999999995e-06 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472862+0000", 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4460000000000003e-06 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501277+0000", 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028414900999999999 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501322+0000", 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4880999999999999e-05 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501358+0000", 2024-09-16T11:44:33.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6245000000000003e-05 2024-09-16T11:44:33.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.679 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.679 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19286 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7979008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.679 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:33.679 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.198642272999997, 2024-09-16T11:44:33.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521860999999999, 2024-09-16T11:44:33.679 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.679 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19286 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473211+0000", 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473214+0000", 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1080000000000001e-06 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473226+0000", 2024-09-16T11:44:33.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2441e-05 2024-09-16T11:44:33.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473234+0000", 2024-09-16T11:44:33.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729999999999998e-06 2024-09-16T11:44:33.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501592+0000", 2024-09-16T11:44:33.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028358491999999999 2024-09-16T11:44:33.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501671+0000", 2024-09-16T11:44:33.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8846999999999999e-05 2024-09-16T11:44:33.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501732+0000", 2024-09-16T11:44:33.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1118000000000007e-05 2024-09-16T11:44:33.685 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.685 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.685 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.686 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20619 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7610368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.686 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:33.686 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.111293105000001, 2024-09-16T11:44:33.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028500152000000001, 2024-09-16T11:44:33.686 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.687 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.687 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20619 2024-09-16T11:44:33.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.688 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:33.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:33.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560560+0000", 2024-09-16T11:44:33.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9499999999999998e-07 2024-09-16T11:44:33.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560561+0000", 2024-09-16T11:44:33.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.071e-06 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560565+0000", 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560567+0000", 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588949+0000", 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381878999999999 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588989+0000", 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0649000000000003e-05 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589059+0000", 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0239000000000003e-05 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.694 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20620 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7716864~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:33.694 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:33.694 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.111283704999998, 2024-09-16T11:44:33.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028627621999999998, 2024-09-16T11:44:33.694 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.694 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.694 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20620 2024-09-16T11:44:33.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.695 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:33.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562377+0000", 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018081729999999999 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562380+0000", 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0469999999999998e-06 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562386+0000", 2024-09-16T11:44:33.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2040000000000002e-06 2024-09-16T11:44:33.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562390+0000", 2024-09-16T11:44:33.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:44:33.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589092+0000", 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026701798999999998 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589139+0000", 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7320000000000001e-05 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589196+0000", 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7201999999999999e-05 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.702 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.703 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41954 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.703 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:33.703 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257577914999999, 2024-09-16T11:44:33.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028594441000000002, 2024-09-16T11:44:33.703 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.704 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.704 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41954 2024-09-16T11:44:33.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.704 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414275+0000", 2024-09-16T11:44:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:44:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414277+0000", 2024-09-16T11:44:33.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:33.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414282+0000", 2024-09-16T11:44:33.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7559999999999996e-06 2024-09-16T11:44:33.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414286+0000", 2024-09-16T11:44:33.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:44:33.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442778+0000", 2024-09-16T11:44:33.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028492237 2024-09-16T11:44:33.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442816+0000", 2024-09-16T11:44:33.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8198000000000003e-05 2024-09-16T11:44:33.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442869+0000", 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2589000000000001e-05 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41955 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257562857, 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028687865, 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.708 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41955 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:33.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:33.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:44:33.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414292+0000", 2024-09-16T11:44:33.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-09-16T11:44:33.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414299+0000", 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4279999999999999e-06 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414303+0000", 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442892+0000", 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028589564000000001 2024-09-16T11:44:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442930+0000", 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330999999999999e-05 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442977+0000", 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7874000000000003e-05 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41956 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257527540000002, 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028766896, 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41956 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:33.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:33.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414326+0000", 2024-09-16T11:44:33.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9900000000000009e-07 2024-09-16T11:44:33.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414330+0000", 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2339999999999996e-06 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414341+0000", 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1358999999999999e-05 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414347+0000", 2024-09-16T11:44:33.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8100000000000003e-06 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443002+0000", 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654331000000002 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443040+0000", 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8262000000000001e-05 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443092+0000", 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1901000000000003e-05 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41957 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257500121, 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028848318000000001, 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41957 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414353+0000", 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414355+0000", 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3489999999999999e-06 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414364+0000", 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1449999999999997e-06 2024-09-16T11:44:33.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414368+0000", 2024-09-16T11:44:33.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8940000000000003e-06 2024-09-16T11:44:33.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443115+0000", 2024-09-16T11:44:33.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028746532000000002 2024-09-16T11:44:33.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443152+0000", 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6814999999999999e-05 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443201+0000", 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9069000000000001e-05 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41958 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:33.721 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257476339, 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028939856, 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41958 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414377+0000", 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4099999999999999e-07 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414379+0000", 2024-09-16T11:44:33.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.418e-06 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414385+0000", 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999999e-06 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414389+0000", 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7740000000000002e-06 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443223+0000", 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028833919 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443263+0000", 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0148999999999998e-05 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443316+0000", 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2636000000000002e-05 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.725 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41959 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257459413999999, 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029040309, 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41959 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.726 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:33.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414395+0000", 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7579999999999999e-06 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414401+0000", 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5330000000000002e-06 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414404+0000", 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.529e-06 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443339+0000", 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028934933999999999 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443383+0000", 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3832000000000003e-05 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443433+0000", 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0309000000000002e-05 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41960 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257444580000001, 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029138134, 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.730 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41960 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:33.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:33.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-07 2024-09-16T11:44:33.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414410+0000", 2024-09-16T11:44:33.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:44:33.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414417+0000", 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414420+0000", 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443457+0000", 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029037065000000001 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443495+0000", 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7555000000000002e-05 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443546+0000", 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1422000000000003e-05 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.734 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41961 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.735 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:33.735 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257428599000001, 2024-09-16T11:44:33.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029230973, 2024-09-16T11:44:33.735 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.735 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.735 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.735 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.735 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.735 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41961 2024-09-16T11:44:33.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.735 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4499999999999998e-07 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414432+0000", 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.413e-06 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414439+0000", 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4769999999999997e-06 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414443+0000", 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8859999999999997e-06 2024-09-16T11:44:33.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443569+0000", 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029126124 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443606+0000", 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6588999999999998e-05 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443655+0000", 2024-09-16T11:44:33.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9138999999999998e-05 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41962 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257405632000001, 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029324505000000001, 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.739 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41962 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414449+0000", 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0279999999999999e-06 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414456+0000", 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3339999999999998e-06 2024-09-16T11:44:33.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414459+0000", 2024-09-16T11:44:33.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.382e-06 2024-09-16T11:44:33.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443678+0000", 2024-09-16T11:44:33.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029218958999999999 2024-09-16T11:44:33.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443719+0000", 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0722000000000001e-05 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443771+0000", 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2661000000000003e-05 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41963 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:33.743 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257389729, 2024-09-16T11:44:33.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029416557999999999, 2024-09-16T11:44:33.744 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.744 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.744 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.744 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.744 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.744 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41963 2024-09-16T11:44:33.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.744 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:33.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:33.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:33.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-16T11:44:33.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414465+0000", 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414470+0000", 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4750000000000001e-06 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414474+0000", 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3720000000000001e-06 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443795+0000", 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029320821 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443830+0000", 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5463000000000002e-05 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443879+0000", 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9147000000000002e-05 2024-09-16T11:44:33.747 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41964 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257374841000001, 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029521202999999999, 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.748 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41964 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:33.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:44:33.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414480+0000", 2024-09-16T11:44:33.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:44:33.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414486+0000", 2024-09-16T11:44:33.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:33.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414489+0000", 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2770000000000001e-06 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443903+0000", 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029414109000000001 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443944+0000", 2024-09-16T11:44:33.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0992999999999999e-05 2024-09-16T11:44:33.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443999+0000", 2024-09-16T11:44:33.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4632000000000001e-05 2024-09-16T11:44:33.752 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.752 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.752 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.752 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41965 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.752 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:33.752 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257359878999999, 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029611293, 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41965 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414495+0000", 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.88e-06 2024-09-16T11:44:33.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414501+0000", 2024-09-16T11:44:33.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2489999999999998e-06 2024-09-16T11:44:33.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414504+0000", 2024-09-16T11:44:33.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3139999999999999e-06 2024-09-16T11:44:33.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444023+0000", 2024-09-16T11:44:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029518564000000001 2024-09-16T11:44:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444058+0000", 2024-09-16T11:44:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5196e-05 2024-09-16T11:44:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444104+0000", 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5732999999999997e-05 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41966 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257343673000001, 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029707678000000001, 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41966 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414514+0000", 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414518+0000", 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3839999999999999e-06 2024-09-16T11:44:33.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414521+0000", 2024-09-16T11:44:33.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0400000000000001e-06 2024-09-16T11:44:33.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444128+0000", 2024-09-16T11:44:33.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029606612000000001 2024-09-16T11:44:33.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444167+0000", 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9418000000000001e-05 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444216+0000", 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9354999999999998e-05 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41967 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:33.761 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257327864000001, 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029796916, 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41967 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6699999999999999e-07 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414527+0000", 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7060000000000001e-06 2024-09-16T11:44:33.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414533+0000", 2024-09-16T11:44:33.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2140000000000001e-06 2024-09-16T11:44:33.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414536+0000", 2024-09-16T11:44:33.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:33.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444239+0000", 2024-09-16T11:44:33.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029702698999999999 2024-09-16T11:44:33.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444273+0000", 2024-09-16T11:44:33.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4229e-05 2024-09-16T11:44:33.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444321+0000", 2024-09-16T11:44:33.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8477999999999999e-05 2024-09-16T11:44:33.765 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.765 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41968 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257312967000001, 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029902765000000001, 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41968 2024-09-16T11:44:33.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414540+0000", 2024-09-16T11:44:33.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:33.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414542+0000", 2024-09-16T11:44:33.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:44:33.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414547+0000", 2024-09-16T11:44:33.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4639999999999997e-06 2024-09-16T11:44:33.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414551+0000", 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6279999999999998e-06 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444346+0000", 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029794885 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444389+0000", 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3442000000000002e-05 2024-09-16T11:44:33.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444442+0000", 2024-09-16T11:44:33.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3173000000000001e-05 2024-09-16T11:44:33.770 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.770 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.770 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.770 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41969 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:33.770 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:33.770 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.257298055, 2024-09-16T11:44:33.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029997612999999999, 2024-09-16T11:44:33.771 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:33.771 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:33.771 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:33.771 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:33.771 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:33.771 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41969 2024-09-16T11:44:33.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.771 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:33.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:33.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:33.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414555+0000", 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414556+0000", 2024-09-16T11:44:33.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.725e-06 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414563+0000", 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2530000000000001e-06 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414566+0000", 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4429999999999999e-06 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444463+0000", 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029897271 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444503+0000", 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9782e-05 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444552+0000", 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8789000000000003e-05 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:33.775 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:33.776 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:33.777 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:33.775+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:33.874 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:33.874 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:33.874 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:33.875 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:33.877 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:33.875+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:33.883 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:33.977 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:33.976+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:33.980 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:33.980 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:33.980 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:33.989 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:34.010 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T11:44:34.030 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:44:34.061 INFO:teuthology.orchestra.run.smithi070.stdout:leaving image image_client.0-clone39 intact 2024-09-16T11:44:34.104 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:34.103+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:34.128 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:34.128 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:34.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:34.129 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:34.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.129 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:34.129 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:34.129 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.205505406, 2024-09-16T11:44:34.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:44:34.129 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.129 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:34.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:34.130 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:34.130 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:44:34.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.130 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:34.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:34.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:34.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:44:34.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:44:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:44:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:44:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:44:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:44:34.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:44:34.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:44:34.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:44:34.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:44:34.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:34.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:44:34.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:44:34.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:34.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:44:34.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:44:34.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:34.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:44:34.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:44:34.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:44:34.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:44:34.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:44:34.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:44:34.135 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.135 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.135 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.136 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:34.136 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:34.136 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.205459714, 2024-09-16T11:44:34.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:44:34.136 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.136 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:34.137 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:34.137 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:34.137 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:44:34.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.137 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:34.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:44:34.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:44:34.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:34.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:44:34.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:44:34.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:34.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:44:34.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:44:34.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:34.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:44:34.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:44:34.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:44:34.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:44:34.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.205402784, 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:44:34.142 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:34.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:44:34.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:34.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:44:34.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:44:34.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:44:34.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:44:34.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:44:34.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:44:34.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:44:34.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:44:34.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:44:34.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:44:34.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:34.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:44:34.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:44:34.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:34.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:44:34.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:44:34.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:34.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:44:34.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:44:34.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:44:34.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:44:34.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:44:34.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:44:34.148 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.148 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.205377416000005, 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:44:34.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:44:34.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:44:34.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:44:34.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:44:34.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:44:34.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:44:34.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:34.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:44:34.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:44:34.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:34.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:44:34.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:44:34.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:34.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:44:34.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:44:34.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:44:34.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:44:34.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:44:34.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:44:34.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.155 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.155 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:34.155 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:34.155 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.205325447999996, 2024-09-16T11:44:34.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:44:34.155 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.155 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:34.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:34.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:34.156 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:44:34.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.156 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:34.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:44:34.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:44:34.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:44:34.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:44:34.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:44:34.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:44:34.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:44:34.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:44:34.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:44:34.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:34.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:44:34.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:44:34.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:34.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:44:34.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:44:34.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:34.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:44:34.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:44:34.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:44:34.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:44:34.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:44:34.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:44:34.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.161 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.205244338, 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:44:34.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:44:34.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:44:34.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:44:34.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:44:34.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:44:34.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:44:34.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:44:34.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:44:34.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:44:34.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:44:34.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:44:34.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.168 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:34.168 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:34.168 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.205218360000003, 2024-09-16T11:44:34.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:44:34.168 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.168 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:34.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:34.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:34.168 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:44:34.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.169 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:34.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:34.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:34.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:44:34.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:44:34.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:44:34.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:44:34.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:44:34.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:34.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:44:34.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:44:34.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:34.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:44:34.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:44:34.174 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.174 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.174 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.174 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:34.174 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:34.174 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.787699735000004, 2024-09-16T11:44:34.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:44:34.174 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:34.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:44:34.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:44:34.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:44:34.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:44:34.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:44:34.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:44:34.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:44:34.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:44:34.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:34.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:44:34.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:44:34.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:34.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:44:34.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:44:34.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:44:34.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:44:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:44:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:44:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:34.180 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.787677803999998, 2024-09-16T11:44:34.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:44:34.181 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.181 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:34.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:34.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:34.181 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:44:34.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.181 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:34.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:34.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:44:34.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:44:34.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:34.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:44:34.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:44:34.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:34.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:44:34.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:44:34.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:34.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:44:34.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:44:34.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:44:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:44:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:44:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:44:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.186 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:34.187 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:34.187 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.787600916000002, 2024-09-16T11:44:34.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:44:34.187 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.187 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:34.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:34.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:34.187 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:44:34.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.187 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:34.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:34.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:44:34.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:44:34.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:44:34.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:44:34.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:44:34.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:44:34.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:34.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:44:34.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:44:34.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:34.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:44:34.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:44:34.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:34.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:44:34.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:44:34.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:44:34.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:44:34.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:44:34.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:44:34.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.193 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.193 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28187 4.17 e52/28 4:ee916bd9:::rbd_data.11c1ce740b46.0000000000000000:head v 52'195, mlcod=52'193)", 2024-09-16T11:44:34.193 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:34.193 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.616181529999999, 2024-09-16T11:44:34.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075854429999999999, 2024-09-16T11:44:34.194 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.194 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.194 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:34.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:34.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509349+0000", 2024-09-16T11:44:34.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2300000000000001e-06 2024-09-16T11:44:34.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513315+0000", 2024-09-16T11:44:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039660590000000001 2024-09-16T11:44:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513328+0000", 2024-09-16T11:44:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.308e-05 2024-09-16T11:44:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513334+0000", 2024-09-16T11:44:34.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:34.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513385+0000", 2024-09-16T11:44:34.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1347000000000002e-05 2024-09-16T11:44:34.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513407+0000", 2024-09-16T11:44:34.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1852999999999999e-05 2024-09-16T11:44:34.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516882+0000", 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034749690000000001 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516933+0000", 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0952999999999998e-05 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28959 4.8 e53/28 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head v 53'408, mlcod=53'406)", 2024-09-16T11:44:34.198 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:34.199 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.094032474000002, 2024-09-16T11:44:34.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088740420000000004, 2024-09-16T11:44:34.199 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.199 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.199 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:34.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031498+0000", 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.652e-06 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033515+0000", 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020165930000000001 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033539+0000", 2024-09-16T11:44:34.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4295999999999999e-05 2024-09-16T11:44:34.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033548+0000", 2024-09-16T11:44:34.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1239999999999994e-06 2024-09-16T11:44:34.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033610+0000", 2024-09-16T11:44:34.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1371999999999994e-05 2024-09-16T11:44:34.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033637+0000", 2024-09-16T11:44:34.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7098999999999999e-05 2024-09-16T11:44:34.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040341+0000", 2024-09-16T11:44:34.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067045819999999997 2024-09-16T11:44:34.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040371+0000", 2024-09-16T11:44:34.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9323999999999999e-05 2024-09-16T11:44:34.203 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.203 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.203 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.203 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32303 4.8 e53/28 4:1436ac82:::rbd_data.11c1cab081b3.0000000000000022:head v 53'417, mlcod=53'415)", 2024-09-16T11:44:34.203 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:34.203 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.623606500000001, 2024-09-16T11:44:34.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085565160000000001, 2024-09-16T11:44:34.204 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.204 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.204 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:34.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:34.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:34.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-09-16T11:44:34.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501930+0000", 2024-09-16T11:44:34.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7649999999999996e-06 2024-09-16T11:44:34.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501946+0000", 2024-09-16T11:44:34.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5301000000000001e-05 2024-09-16T11:44:34.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501952+0000", 2024-09-16T11:44:34.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:44:34.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502100+0000", 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014847600000000001 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502120+0000", 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9303000000000001e-05 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510436+0000", 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083166420000000008 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510479+0000", 2024-09-16T11:44:34.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2979000000000003e-05 2024-09-16T11:44:34.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.208 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.208 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32306 4.1e e53/28 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head v 53'124, mlcod=53'122)", 2024-09-16T11:44:34.208 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:34.208 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.623540505999998, 2024-09-16T11:44:34.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084990900000000008, 2024-09-16T11:44:34.208 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.209 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.209 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:34.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:34.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:34.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:44:34.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501994+0000", 2024-09-16T11:44:34.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:44:34.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502002+0000", 2024-09-16T11:44:34.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3839999999999997e-06 2024-09-16T11:44:34.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502007+0000", 2024-09-16T11:44:34.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6290000000000001e-06 2024-09-16T11:44:34.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502454+0000", 2024-09-16T11:44:34.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00044736800000000002 2024-09-16T11:44:34.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502471+0000", 2024-09-16T11:44:34.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6648999999999999e-05 2024-09-16T11:44:34.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510455+0000", 2024-09-16T11:44:34.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0079841909999999999 2024-09-16T11:44:34.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510488+0000", 2024-09-16T11:44:34.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2948999999999999e-05 2024-09-16T11:44:34.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.214 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.214 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32307 4.8 e53/28 4:11c1c37d:::rbd_data.11c1cab081b3.0000000000000026:head v 53'418, mlcod=53'415)", 2024-09-16T11:44:34.214 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:34.214 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 32.622785170999997, 2024-09-16T11:44:34.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00776212, 2024-09-16T11:44:34.215 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.215 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.215 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:34.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:34.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:34.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:34.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502750+0000", 2024-09-16T11:44:34.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:44:34.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502760+0000", 2024-09-16T11:44:34.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:44:34.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502767+0000", 2024-09-16T11:44:34.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4219999999999997e-06 2024-09-16T11:44:34.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502802+0000", 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5089999999999998e-05 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502816+0000", 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4092e-05 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510483+0000", 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00766704 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510506+0000", 2024-09-16T11:44:34.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3195999999999999e-05 2024-09-16T11:44:34.219 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.219 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.219 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.219 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33583 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'416, mlcod=53'411)", 2024-09-16T11:44:34.219 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:34.219 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.914826762000001, 2024-09-16T11:44:34.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078132459999999994, 2024-09-16T11:44:34.219 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.219 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.220 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:34.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:34.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210703+0000", 2024-09-16T11:44:34.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:34.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210706+0000", 2024-09-16T11:44:34.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-06 2024-09-16T11:44:34.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210712+0000", 2024-09-16T11:44:34.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:44:34.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210715+0000", 2024-09-16T11:44:34.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:44:34.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210736+0000", 2024-09-16T11:44:34.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0293999999999999e-05 2024-09-16T11:44:34.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210746+0000", 2024-09-16T11:44:34.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0178e-05 2024-09-16T11:44:34.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218477+0000", 2024-09-16T11:44:34.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007731501 2024-09-16T11:44:34.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218516+0000", 2024-09-16T11:44:34.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8148000000000001e-05 2024-09-16T11:44:34.225 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.225 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.225 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.226 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33590 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'419, mlcod=53'415)", 2024-09-16T11:44:34.226 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:34.226 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.903768621000001, 2024-09-16T11:44:34.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082499879999999998, 2024-09-16T11:44:34.226 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.226 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.227 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:34.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:34.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221761+0000", 2024-09-16T11:44:34.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-16T11:44:34.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221765+0000", 2024-09-16T11:44:34.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1590000000000003e-06 2024-09-16T11:44:34.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221774+0000", 2024-09-16T11:44:34.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7490000000000004e-06 2024-09-16T11:44:34.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221778+0000", 2024-09-16T11:44:34.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0419999999999996e-06 2024-09-16T11:44:34.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221800+0000", 2024-09-16T11:44:34.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1926999999999999e-05 2024-09-16T11:44:34.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221810+0000", 2024-09-16T11:44:34.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0079e-05 2024-09-16T11:44:34.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.229980+0000", 2024-09-16T11:44:34.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081698469999999992 2024-09-16T11:44:34.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230010+0000", 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0746e-05 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33591 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'420, mlcod=53'415)", 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.903207136999999, 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007702576, 2024-09-16T11:44:34.231 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.232 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.232 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:34.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:34.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:34.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:44:34.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222326+0000", 2024-09-16T11:44:34.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:44:34.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222337+0000", 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1008e-05 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222342+0000", 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0050000000000004e-06 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222366+0000", 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3696000000000001e-05 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222385+0000", 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230012+0000", 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076270940000000001 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230025+0000", 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2112e-05 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.235 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.236 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.236 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33593 4.1e e53/28 4:7e77cec2:::rbd_data.11c1cab081b3.000000000000000c:head v 53'133, mlcod=53'131)", 2024-09-16T11:44:34.236 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:34.236 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.898573214999999, 2024-09-16T11:44:34.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070516270000000004, 2024-09-16T11:44:34.236 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.236 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226956+0000", 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.226957+0000", 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227014+0000", 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7074999999999998e-05 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227024+0000", 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0617e-05 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227030+0000", 2024-09-16T11:44:34.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3279999999999996e-06 2024-09-16T11:44:34.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227061+0000", 2024-09-16T11:44:34.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1718999999999999e-05 2024-09-16T11:44:34.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.227074+0000", 2024-09-16T11:44:34.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2738000000000001e-05 2024-09-16T11:44:34.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.233975+0000", 2024-09-16T11:44:34.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0069008480000000002 2024-09-16T11:44:34.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.234008+0000", 2024-09-16T11:44:34.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2614000000000003e-05 2024-09-16T11:44:34.240 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.240 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.240 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.241 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:55282 4.1e e73/28 4:785299e2:::rbd_data.11c11a49c2da.0000000000000001:head v 73'324, mlcod=73'322)", 2024-09-16T11:44:34.241 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:44:34.241 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8865106179999991, 2024-09-16T11:44:34.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071431890000000003, 2024-09-16T11:44:34.241 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:34.241 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:34.241 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:34.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:34.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239020+0000", 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.049e-06 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244238+0000", 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052183489999999997 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244252+0000", 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4246999999999999e-05 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244275+0000", 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3093999999999999e-05 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:34.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244314+0000", 2024-09-16T11:44:34.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8495000000000003e-05 2024-09-16T11:44:34.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:34.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244330+0000", 2024-09-16T11:44:34.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5917e-05 2024-09-16T11:44:34.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:34.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246121+0000", 2024-09-16T11:44:34.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017909239999999999 2024-09-16T11:44:34.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:34.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:34.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:34.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246162+0000", 2024-09-16T11:44:34.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1114e-05 2024-09-16T11:44:34.245 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.245 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.245 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.245 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:34.245 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:34.245 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:34.246 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:34.247 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:34.247 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-16T11:44:34.247 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:34.248 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:34.248 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:34.249 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:34.249 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:34.249 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:34.250 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:34.250 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:34.245+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:34.348 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:34.347+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:34.352 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:34.352 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:34.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:34.368 INFO:teuthology.orchestra.run.smithi070.stdout:5922 write 0x98bbe5b thru 0x98c5238 (0x93de bytes) 2024-09-16T11:44:34.393 INFO:teuthology.orchestra.run.smithi070.stdout:5924 trunc from 0xabc91bd to 0xc990f4e 2024-09-16T11:44:34.398 INFO:teuthology.orchestra.run.smithi070.stdout:5925 writesame 0x832d56e thru 0x8336d6f data_size 0x1(0x9802 bytes) 2024-09-16T11:44:34.434 INFO:teuthology.orchestra.run.smithi070.stdout:5926 writesame 0x46a799 thru 0x46f880 data_size 0xc(0x50e8 bytes) 2024-09-16T11:44:34.449 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:34.448+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:34.466 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:34.466 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:34.476 INFO:teuthology.orchestra.run.smithi070.stdout:5928 punch from 0xb63f876 to 0xb641a57, (0x21e1 bytes) 2024-09-16T11:44:34.476 INFO:teuthology.orchestra.run.smithi070.stdout:5929 read 0x95792c1 thru 0x95802c7 (0x7007 bytes) 2024-09-16T11:44:34.477 INFO:teuthology.orchestra.run.smithi070.stdout:5930 punch from 0x45abaa to 0x45df4b, (0x33a1 bytes) 2024-09-16T11:44:34.477 INFO:teuthology.orchestra.run.smithi070.stdout:5931 write 0x723d8f0 thru 0x724d20b (0xf91c bytes) 2024-09-16T11:44:34.499 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:34.500 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:34.510 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:34.512 INFO:teuthology.orchestra.run.smithi070.stdout:5932 write 0x7ce4d6 thru 0x7de288 (0xfdb3 bytes) 2024-09-16T11:44:34.550 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:34.548+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:34.581 INFO:teuthology.orchestra.run.smithi070.stdout:5933 punch from 0x8ec04d1 to 0x8ec3c62, (0x3791 bytes) 2024-09-16T11:44:34.581 INFO:teuthology.orchestra.run.smithi070.stdout:5934 write 0x5c29b80 thru 0x5c34f80 (0xb401 bytes) 2024-09-16T11:44:34.624 INFO:teuthology.orchestra.run.smithi070.stdout:5936 punch from 0x3d9c529 to 0x3da44a0, (0x7f77 bytes) 2024-09-16T11:44:34.624 INFO:teuthology.orchestra.run.smithi070.stdout:5938 read 0x2c111 thru 0x2db71 (0x1a61 bytes) 2024-09-16T11:44:34.625 INFO:teuthology.orchestra.run.smithi070.stdout:5939 read 0xc7f1dc1 thru 0xc7fb9fd (0x9c3d bytes) 2024-09-16T11:44:34.625 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:34.627 INFO:teuthology.orchestra.run.smithi070.stdout:5943 trunc from 0xc990f4e to 0xebb9ea8 2024-09-16T11:44:34.633 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:34.633 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:34.634 INFO:teuthology.orchestra.run.smithi070.stdout:5948 read 0x4032766 thru 0x40361e6 (0x3a81 bytes) 2024-09-16T11:44:34.638 INFO:teuthology.orchestra.run.smithi070.stdout:5949 write 0xebb4d2c thru 0xebc0a6e (0xbd43 bytes) 2024-09-16T11:44:34.639 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:34.639 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:34.642 INFO:teuthology.orchestra.run.smithi070.stdout:5950 write 0x62c4d2b thru 0x62cc125 (0x73fb bytes) 2024-09-16T11:44:34.650 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:34.649+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:34.651 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:34.669 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:34.677 INFO:teuthology.orchestra.run.smithi070.stdout:5952 trunc from 0xebc0a6f to 0x958e679 2024-09-16T11:44:34.697 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:34.697 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:34.727 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:34.728 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:34.751 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:34.750+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:34.753 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:34.753 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:34.761 INFO:teuthology.orchestra.run.smithi070.stdout:5953 punch from 0x1251886 to 0x1257423, (0x5b9d bytes) 2024-09-16T11:44:34.761 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:34.761 INFO:teuthology.orchestra.run.smithi070.stdout:5955 write 0x76531bb thru 0x765388b (0x6d1 bytes) 2024-09-16T11:44:34.804 INFO:teuthology.orchestra.run.smithi070.stdout:5956 write 0x759c1c2 thru 0x759c508 (0x347 bytes) 2024-09-16T11:44:34.827 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:34.827 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:34.836 INFO:teuthology.orchestra.run.smithi070.stdout:5957 writesame 0x5af0725 thru 0x5af3a05 data_size 0x5(0x32e1 bytes) 2024-09-16T11:44:34.837 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:34.878 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:34.877+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:34.953 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:34.951+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:34.974 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:34.974 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:34.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:35.054 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:35.053+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:35.061 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:35.061 INFO:teuthology.orchestra.run.smithi070.stdout:5959 punch from 0x8eca016 to 0x8ed2493, (0x847d bytes) 2024-09-16T11:44:35.067 INFO:teuthology.orchestra.run.smithi070.stdout:5960 trunc from 0x958e679 to 0x44d3007 2024-09-16T11:44:35.101 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:35.101 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:35.114 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:35.155 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:35.154+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:35.247 INFO:teuthology.orchestra.run.smithi070.stdout:5962 trunc from 0x44d3007 to 0xa822d7 2024-09-16T11:44:35.256 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:35.254+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:35.289 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:35.289 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:35.302 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:35.356 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:35.355+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:35.376 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:44:35.376 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:44:35.376 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T11:44:35.376 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T11:44:35.446 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:35.446 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:35.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:35.457 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:35.455+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:35.558 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:35.556+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:35.573 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:35.573 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:35.583 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:35.587 INFO:teuthology.orchestra.run.smithi070.stdout:5963 write 0x44f33c0 thru 0x44fdf0b (0xab4c bytes) 2024-09-16T11:44:35.606 INFO:teuthology.orchestra.run.smithi070.stdout:5964 writesame 0xc3a94f8 thru 0xc3b3937 data_size 0x92(0xa440 bytes) 2024-09-16T11:44:35.606 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:35.606 INFO:teuthology.orchestra.run.smithi070.stdout:5966 read 0x6be1cde thru 0x6be9318 (0x763b bytes) 2024-09-16T11:44:35.606 INFO:teuthology.orchestra.run.smithi070.stdout:5967 punch from 0x4a166d9 to 0x4a240f5, (0xda1c bytes) 2024-09-16T11:44:35.606 INFO:teuthology.orchestra.run.smithi070.stdout:5968 write 0x14481bb thru 0x144b2e5 (0x312b bytes) 2024-09-16T11:44:35.614 INFO:teuthology.orchestra.run.smithi070.stdout:5971 write 0xd86070d thru 0xd86d474 (0xcd68 bytes) 2024-09-16T11:44:35.618 INFO:teuthology.orchestra.run.smithi070.stdout:5973 punch from 0x8736bd4 to 0x8744ec8, (0xe2f4 bytes) 2024-09-16T11:44:35.618 INFO:teuthology.orchestra.run.smithi070.stdout:5974 read 0x15d0593 thru 0x15dedcc (0xe83a bytes) 2024-09-16T11:44:35.618 INFO:teuthology.orchestra.run.smithi070.stdout:5975 write 0xa0ff498 thru 0xa1019e8 (0x2551 bytes) 2024-09-16T11:44:35.621 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:35.621 INFO:teuthology.orchestra.run.smithi070.stdout:5977 write 0x62808e0 thru 0x628d191 (0xc8b2 bytes) 2024-09-16T11:44:35.624 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:35.629 INFO:teuthology.orchestra.run.smithi070.stdout:5979 trunc from 0xd86d475 to 0x9b5e0a1 2024-09-16T11:44:35.650 INFO:teuthology.orchestra.run.smithi070.stdout:5980 write 0xdf988c6 thru 0xdfa21df (0x991a bytes) 2024-09-16T11:44:35.653 INFO:teuthology.orchestra.run.smithi070.stdout:5981 read 0x1af50e8 thru 0x1af5642 (0x55b bytes) 2024-09-16T11:44:35.653 INFO:teuthology.orchestra.run.smithi070.stdout:5982 write 0x5c2da7e thru 0x5c3b63a (0xdbbd bytes) 2024-09-16T11:44:35.656 INFO:teuthology.orchestra.run.smithi070.stdout:5983 read 0x7d46f3 thru 0x7de2e2 (0x9bf0 bytes) 2024-09-16T11:44:35.658 INFO:teuthology.orchestra.run.smithi070.stdout:5985 write 0x2873ed3 thru 0x2878a9b (0x4bc9 bytes) 2024-09-16T11:44:35.658 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:35.657+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:35.660 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:35.661 INFO:teuthology.orchestra.run.smithi070.stdout:compare and write disabled 2024-09-16T11:44:35.672 INFO:teuthology.orchestra.run.smithi070.stdout:5988 trunc from 0xdfa21e0 to 0x460f940 2024-09-16T11:44:35.683 INFO:teuthology.orchestra.run.smithi070.stdout:5989 read 0x41a0186 thru 0x41abdec (0xbc67 bytes) 2024-09-16T11:44:35.683 INFO:teuthology.orchestra.run.smithi070.stdout:5991 read 0x257b713 thru 0x2587618 (0xbf06 bytes) 2024-09-16T11:44:35.692 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:35.692 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:35.692 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 83, 2024-09-16T11:44:35.692 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "3420bcad-625f-4e15-aaff-fb4236322168", 2024-09-16T11:44:35.692 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2024-09-16T11:43:01.654626+0000", 2024-09-16T11:44:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2024-09-16T11:44:34.003349+0000", 2024-09-16T11:44:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2024-09-16T11:43:11.689589+0000", 2024-09-16T11:44:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2024-09-16T11:43:04.526236+0000", 2024-09-16T11:44:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T11:44:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5806080, 2024-09-16T11:44:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2024-09-16T11:44:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: "nodeep-scrub", 2024-09-16T11:44:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: "noscrub", 2024-09-16T11:44:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2024-09-16T11:44:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2024-09-16T11:44:35.694 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2024-09-16T11:44:35.694 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2024-09-16T11:44:35.694 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:35.694 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 5, 2024-09-16T11:44:35.694 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T11:44:35.694 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T11:44:35.694 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T11:44:35.694 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2024-09-16T11:44:35.694 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 4, 2024-09-16T11:44:35.694 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2024-09-16T11:44:35.694 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2024-09-16T11:44:35.695 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2024-09-16T11:44:35.695 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "quincy", 2024-09-16T11:44:35.695 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2024-09-16T11:44:35.695 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.695 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2024-09-16T11:44:35.695 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2024-09-16T11:44:35.695 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T11:43:11.406582+0000", 2024-09-16T11:44:35.695 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T11:44:35.695 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T11:44:35.695 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T11:44:35.695 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-16T11:44:35.696 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-16T11:44:35.696 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T11:44:35.696 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:44:35.696 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:44:35.696 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:44:35.696 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:44:35.696 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T11:44:35.696 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:44:35.696 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2024-09-16T11:44:35.696 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2024-09-16T11:44:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2024-09-16T11:44:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2024-09-16T11:44:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2024-09-16T11:44:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T11:44:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T11:44:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T11:44:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T11:44:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T11:44:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T11:44:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T11:44:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.698 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "16", 2024-09-16T11:44:35.698 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T11:44:35.698 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:44:35.698 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:44:35.698 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T11:44:35.698 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:44:35.698 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T11:44:35.698 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T11:44:35.698 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T11:44:35.698 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T11:44:35.698 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T11:44:35.698 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T11:44:35.699 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T11:44:35.699 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T11:44:35.699 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T11:44:35.699 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T11:44:35.699 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T11:44:35.699 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T11:44:35.699 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T11:44:35.699 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:44:35.699 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:44:35.699 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:44:35.699 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T11:44:35.700 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T11:44:35.700 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T11:44:35.700 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T11:44:35.700 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T11:44:35.700 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.700 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T11:44:35.700 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T11:44:35.700 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T11:44:35.700 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:44:35.700 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:44:35.700 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:44:35.701 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:44:35.701 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T11:44:35.701 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T11:44:35.701 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T11:44:35.701 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T11:44:35.701 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T11:44:35.701 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2024-09-16T11:44:35.701 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2024-09-16T11:44:35.701 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.701 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T11:44:35.701 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2024-09-16T11:44:35.702 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.702 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.702 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.702 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2024-09-16T11:44:35.702 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2024-09-16T11:44:35.702 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T11:43:13.125709+0000", 2024-09-16T11:44:35.702 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T11:44:35.702 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T11:44:35.702 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T11:44:35.702 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-16T11:44:35.702 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-16T11:44:35.703 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T11:44:35.703 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:44:35.703 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:44:35.703 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:44:35.703 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:44:35.703 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T11:44:35.703 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:44:35.703 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2024-09-16T11:44:35.703 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2024-09-16T11:44:35.703 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2024-09-16T11:44:35.703 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2024-09-16T11:44:35.703 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2024-09-16T11:44:35.704 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T11:44:35.704 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T11:44:35.704 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T11:44:35.704 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T11:44:35.704 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T11:44:35.704 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T11:44:35.704 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T11:44:35.704 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.704 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "23", 2024-09-16T11:44:35.704 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "23", 2024-09-16T11:44:35.704 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "23", 2024-09-16T11:44:35.705 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "23", 2024-09-16T11:44:35.705 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T11:44:35.705 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:44:35.705 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 3, 2024-09-16T11:44:35.705 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 21, 2024-09-16T11:44:35.705 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T11:44:35.705 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T11:44:35.705 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T11:44:35.705 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T11:44:35.705 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [ 2024-09-16T11:44:35.705 INFO:teuthology.orchestra.run.smithi070.stdout: 3 2024-09-16T11:44:35.706 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:35.706 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T11:44:35.706 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": 3, 2024-09-16T11:44:35.706 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": 3, 2024-09-16T11:44:35.706 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T11:44:35.706 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T11:44:35.706 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T11:44:35.706 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:44:35.706 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:44:35.706 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:44:35.706 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T11:44:35.706 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T11:44:35.707 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T11:44:35.707 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T11:44:35.707 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T11:44:35.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.707 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T11:44:35.707 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T11:44:35.707 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T11:44:35.707 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:44:35.707 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:44:35.707 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:44:35.707 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:44:35.708 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T11:44:35.708 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T11:44:35.708 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T11:44:35.708 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T11:44:35.708 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T11:44:35.708 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T11:44:35.708 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T11:44:35.708 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.708 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.708 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.708 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 3, 2024-09-16T11:44:35.708 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cache", 2024-09-16T11:44:35.709 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T11:43:21.227888+0000", 2024-09-16T11:44:35.709 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8201, 2024-09-16T11:44:35.709 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,incomplete_clones,selfmanaged_snaps", 2024-09-16T11:44:35.709 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T11:44:35.709 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-16T11:44:35.709 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-16T11:44:35.709 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T11:44:35.709 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:44:35.709 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:44:35.709 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:44:35.709 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:44:35.709 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T11:44:35.710 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:44:35.710 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 4, 2024-09-16T11:44:35.710 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 4, 2024-09-16T11:44:35.710 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 4, 2024-09-16T11:44:35.710 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 4, 2024-09-16T11:44:35.710 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 4, 2024-09-16T11:44:35.710 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T11:44:35.710 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T11:44:35.710 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T11:44:35.710 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T11:44:35.711 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T11:44:35.711 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T11:44:35.711 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T11:44:35.711 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.711 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "27", 2024-09-16T11:44:35.711 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "23", 2024-09-16T11:44:35.711 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "23", 2024-09-16T11:44:35.711 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "23", 2024-09-16T11:44:35.712 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T11:44:35.712 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:44:35.712 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 3, 2024-09-16T11:44:35.712 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 21, 2024-09-16T11:44:35.712 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T11:44:35.712 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T11:44:35.712 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T11:44:35.712 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T11:44:35.712 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T11:44:35.712 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": 2, 2024-09-16T11:44:35.712 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T11:44:35.713 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T11:44:35.713 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "writeback", 2024-09-16T11:44:35.713 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T11:44:35.713 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 250, 2024-09-16T11:44:35.713 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:44:35.713 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:44:35.713 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:44:35.713 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T11:44:35.713 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T11:44:35.713 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T11:44:35.714 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T11:44:35.714 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "bloom", 2024-09-16T11:44:35.714 INFO:teuthology.orchestra.run.smithi070.stdout: "false_positive_probability": 0.050000000000000003, 2024-09-16T11:44:35.714 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size": 0, 2024-09-16T11:44:35.714 INFO:teuthology.orchestra.run.smithi070.stdout: "seed": 0 2024-09-16T11:44:35.714 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.714 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 60, 2024-09-16T11:44:35.714 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 8, 2024-09-16T11:44:35.714 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T11:44:35.714 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:44:35.714 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:44:35.715 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:44:35.715 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:44:35.715 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [ 2024-09-16T11:44:35.715 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:35.715 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:35.715 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:35.715 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:35.715 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:35.715 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:35.715 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000, 2024-09-16T11:44:35.715 INFO:teuthology.orchestra.run.smithi070.stdout: 1000000 2024-09-16T11:44:35.716 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:35.716 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T11:44:35.716 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T11:44:35.716 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T11:44:35.716 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T11:44:35.716 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T11:44:35.716 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T11:44:35.716 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.716 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.716 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 4, 2024-09-16T11:44:35.717 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "pool_client.0", 2024-09-16T11:44:35.717 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T11:43:30.105533+0000", 2024-09-16T11:44:35.717 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T11:44:35.717 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T11:44:35.717 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T11:44:35.717 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-16T11:44:35.717 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-16T11:44:35.717 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T11:44:35.717 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:44:35.717 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:44:35.717 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:44:35.718 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:44:35.718 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T11:44:35.718 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:44:35.718 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T11:44:35.718 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T11:44:35.718 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T11:44:35.718 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T11:44:35.718 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T11:44:35.718 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T11:44:35.718 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T11:44:35.718 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T11:44:35.719 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T11:44:35.719 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T11:44:35.719 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T11:44:35.719 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T11:44:35.719 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.719 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "83", 2024-09-16T11:44:35.719 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T11:44:35.719 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:44:35.719 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:44:35.719 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T11:44:35.719 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:44:35.719 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 43, 2024-09-16T11:44:35.720 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 83, 2024-09-16T11:44:35.720 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T11:44:35.720 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T11:44:35.720 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T11:44:35.720 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T11:44:35.720 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T11:44:35.720 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T11:44:35.720 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T11:44:35.720 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T11:44:35.720 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T11:44:35.720 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T11:44:35.721 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T11:44:35.721 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:44:35.721 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:44:35.721 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:44:35.721 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T11:44:35.721 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T11:44:35.721 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T11:44:35.721 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T11:44:35.721 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T11:44:35.721 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.721 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T11:44:35.722 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T11:44:35.722 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T11:44:35.722 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:44:35.722 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:44:35.722 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:44:35.722 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:44:35.722 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T11:44:35.722 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T11:44:35.722 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T11:44:35.722 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T11:44:35.722 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T11:44:35.723 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T11:44:35.723 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T11:44:35.723 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.723 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.723 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:35.723 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T11:44:35.723 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.723 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T11:44:35.723 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "91f052e4-4c8f-4484-92ce-bd1a7f8f3efb", 2024-09-16T11:44:35.723 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:35.723 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:35.724 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:35.724 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:35.724 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:35.724 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:35.724 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T11:44:35.724 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:35.724 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:35.724 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:35.724 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:35.724 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.724 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.724 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.725 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2024-09-16T11:44:35.725 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:35.725 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.725 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.725 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.725 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2024-09-16T11:44:35.725 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:35.725 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.725 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.725 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.725 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:35.726 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.726 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.726 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.726 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2024-09-16T11:44:35.726 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:35.726 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.726 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.726 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.726 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2024-09-16T11:44:35.726 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:35.726 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.726 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.727 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:35.727 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.727 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.727 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.727 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2024-09-16T11:44:35.727 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:35.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.727 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.727 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.727 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2024-09-16T11:44:35.727 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:35.727 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.728 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.728 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:35.728 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.728 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.728 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2024-09-16T11:44:35.728 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:35.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.729 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.729 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2024-09-16T11:44:35.729 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33355 2024-09-16T11:44:35.729 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.729 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.729 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/33355", 2024-09-16T11:44:35.729 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/33355", 2024-09-16T11:44:35.729 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/33355", 2024-09-16T11:44:35.729 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/33355", 2024-09-16T11:44:35.730 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:35.730 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:35.730 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:35.730 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.730 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.730 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T11:44:35.730 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "568b1280-7365-4094-afe8-b0b5e766de6f", 2024-09-16T11:44:35.730 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:35.730 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:35.731 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:35.731 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:35.731 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:35.731 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:35.731 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 14, 2024-09-16T11:44:35.731 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 61, 2024-09-16T11:44:35.731 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:35.731 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:35.731 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:35.731 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.731 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.731 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2024-09-16T11:44:35.732 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:35.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.732 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.732 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2024-09-16T11:44:35.732 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:35.732 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.732 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.732 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:35.732 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.733 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.733 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2024-09-16T11:44:35.733 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:35.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.733 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.733 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2024-09-16T11:44:35.733 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:35.733 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.733 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.734 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:35.734 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.734 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.734 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2024-09-16T11:44:35.734 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:35.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.734 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.734 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2024-09-16T11:44:35.734 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:35.735 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.735 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.735 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:35.735 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.735 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.735 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2024-09-16T11:44:35.735 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:35.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.736 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.736 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2024-09-16T11:44:35.736 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33359 2024-09-16T11:44:35.736 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.736 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.736 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/33359", 2024-09-16T11:44:35.736 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/33359", 2024-09-16T11:44:35.736 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/33359", 2024-09-16T11:44:35.736 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/33359", 2024-09-16T11:44:35.736 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:35.737 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:35.737 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:35.737 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.737 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T11:44:35.737 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "328adfa3-c1b4-4745-9759-7327c8cbd499", 2024-09-16T11:44:35.737 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:35.737 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:35.737 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:35.737 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:35.738 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:35.738 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:35.738 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T11:44:35.738 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:35.738 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:35.738 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:35.738 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:35.738 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.738 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.738 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2024-09-16T11:44:35.739 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:35.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.739 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.739 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2024-09-16T11:44:35.739 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:35.739 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.739 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.739 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:35.739 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.740 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.740 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.740 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2024-09-16T11:44:35.740 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:35.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.740 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.740 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.740 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2024-09-16T11:44:35.740 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:35.740 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.740 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.741 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:35.741 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.741 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.741 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2024-09-16T11:44:35.741 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:35.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.741 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.741 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2024-09-16T11:44:35.742 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:35.742 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.742 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.742 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:35.742 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.742 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.742 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2024-09-16T11:44:35.742 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:35.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.743 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.743 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2024-09-16T11:44:35.743 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33356 2024-09-16T11:44:35.743 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.743 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.743 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/33356", 2024-09-16T11:44:35.743 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/33356", 2024-09-16T11:44:35.743 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6815/33356", 2024-09-16T11:44:35.743 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/33356", 2024-09-16T11:44:35.744 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:35.744 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:35.744 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:35.744 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.744 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T11:44:35.744 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "6df66ae7-e835-4d8f-bd27-243d7a9705b5", 2024-09-16T11:44:35.744 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:35.744 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:35.744 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:35.744 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:35.745 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:35.745 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:35.745 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 14, 2024-09-16T11:44:35.745 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 77, 2024-09-16T11:44:35.745 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:35.745 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:35.745 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:35.746 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.746 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.746 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2024-09-16T11:44:35.746 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:35.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.746 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.746 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2024-09-16T11:44:35.746 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:35.746 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.747 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.747 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:35.747 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.747 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.747 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2024-09-16T11:44:35.747 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:35.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.747 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.748 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2024-09-16T11:44:35.748 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:35.748 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.748 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.748 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:35.748 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.748 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.748 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2024-09-16T11:44:35.748 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:35.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.749 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.749 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2024-09-16T11:44:35.749 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:35.749 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.749 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.749 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:35.749 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.750 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.750 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2024-09-16T11:44:35.750 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:35.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.750 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.750 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.750 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2024-09-16T11:44:35.750 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33384 2024-09-16T11:44:35.750 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.750 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.750 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/33384", 2024-09-16T11:44:35.751 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/33384", 2024-09-16T11:44:35.751 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/33384", 2024-09-16T11:44:35.751 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/33384", 2024-09-16T11:44:35.751 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:35.751 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:35.751 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:35.751 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.752 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.752 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T11:44:35.752 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "29f52d62-d4c3-49f9-992f-23817f5622d2", 2024-09-16T11:44:35.752 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:35.752 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:35.752 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:35.752 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:35.752 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:35.752 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:35.752 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T11:44:35.753 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:35.753 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:35.753 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:35.753 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:35.753 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.753 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.753 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.753 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6800", 2024-09-16T11:44:35.753 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:35.753 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.754 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.754 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6803", 2024-09-16T11:44:35.754 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:35.754 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.754 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.754 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:35.754 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.754 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.754 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6805", 2024-09-16T11:44:35.755 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:35.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.755 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.755 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.755 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6807", 2024-09-16T11:44:35.755 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:35.755 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.755 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.755 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:35.756 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.756 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.756 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6813", 2024-09-16T11:44:35.756 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:35.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.756 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.756 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6815", 2024-09-16T11:44:35.756 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:35.756 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.756 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.757 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:35.757 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.757 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.757 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6809", 2024-09-16T11:44:35.757 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:35.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.757 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.757 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6811", 2024-09-16T11:44:35.757 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33331 2024-09-16T11:44:35.758 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.758 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.758 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.192:6803/33331", 2024-09-16T11:44:35.758 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.192:6807/33331", 2024-09-16T11:44:35.758 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.192:6815/33331", 2024-09-16T11:44:35.758 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.192:6811/33331", 2024-09-16T11:44:35.758 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:35.758 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:35.758 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:35.758 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.759 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.759 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T11:44:35.759 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "0c1a275a-3c75-4e0b-bd35-d34b64d99c0a", 2024-09-16T11:44:35.759 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:35.759 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:35.759 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:35.759 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:35.759 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:35.759 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:35.759 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 14, 2024-09-16T11:44:35.759 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 60, 2024-09-16T11:44:35.760 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:35.760 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:35.760 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:35.760 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.760 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.760 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.760 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6816", 2024-09-16T11:44:35.760 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:35.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.760 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.760 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.761 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6817", 2024-09-16T11:44:35.761 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:35.761 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.761 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.761 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.761 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:35.761 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.761 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.761 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.761 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6818", 2024-09-16T11:44:35.761 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:35.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.762 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.762 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6819", 2024-09-16T11:44:35.762 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:35.762 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.762 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.762 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:35.762 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.763 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.763 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6822", 2024-09-16T11:44:35.763 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:35.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.763 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.763 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6823", 2024-09-16T11:44:35.764 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:35.764 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.764 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.764 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.764 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:35.764 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.764 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.764 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.764 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6820", 2024-09-16T11:44:35.764 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:35.764 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.765 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.765 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6821", 2024-09-16T11:44:35.765 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33333 2024-09-16T11:44:35.765 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.765 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.765 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.192:6817/33333", 2024-09-16T11:44:35.765 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.192:6819/33333", 2024-09-16T11:44:35.765 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.192:6823/33333", 2024-09-16T11:44:35.765 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.192:6821/33333", 2024-09-16T11:44:35.766 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:35.766 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:35.766 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:35.766 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T11:44:35.766 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "d79f0070-ce58-48e7-a56a-2762ec76a271", 2024-09-16T11:44:35.766 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:35.766 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:35.766 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:35.767 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:35.767 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:35.767 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:35.767 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T11:44:35.767 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-16T11:44:35.767 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:35.767 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:35.767 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:35.767 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.767 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.767 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6801", 2024-09-16T11:44:35.768 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:35.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.768 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.768 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6802", 2024-09-16T11:44:35.768 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:35.768 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.768 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.768 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:35.768 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.769 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.769 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6804", 2024-09-16T11:44:35.769 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:35.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.769 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.769 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6806", 2024-09-16T11:44:35.769 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:35.769 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.769 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.770 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:35.770 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.770 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.770 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6812", 2024-09-16T11:44:35.770 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:35.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.770 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.770 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6814", 2024-09-16T11:44:35.770 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:35.771 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.771 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.771 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:35.771 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.771 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.771 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6808", 2024-09-16T11:44:35.771 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:35.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.771 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.772 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6810", 2024-09-16T11:44:35.772 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33332 2024-09-16T11:44:35.772 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.772 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.772 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.192:6802/33332", 2024-09-16T11:44:35.772 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.192:6806/33332", 2024-09-16T11:44:35.772 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.192:6814/33332", 2024-09-16T11:44:35.772 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.192:6810/33332", 2024-09-16T11:44:35.772 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:35.772 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:35.773 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:35.773 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T11:44:35.773 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "537fa5df-3d0a-4dd4-8155-20ee94e41c82", 2024-09-16T11:44:35.773 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T11:44:35.773 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T11:44:35.773 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T11:44:35.773 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T11:44:35.773 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T11:44:35.774 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T11:44:35.774 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 14, 2024-09-16T11:44:35.774 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 16, 2024-09-16T11:44:35.774 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T11:44:35.774 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T11:44:35.774 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T11:44:35.774 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.774 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.774 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.774 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6824", 2024-09-16T11:44:35.774 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:35.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.775 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.775 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6825", 2024-09-16T11:44:35.775 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:35.775 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.775 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.775 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T11:44:35.775 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.775 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.775 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6826", 2024-09-16T11:44:35.776 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:35.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.776 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.776 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6827", 2024-09-16T11:44:35.776 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:35.776 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.776 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.776 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T11:44:35.776 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.777 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.777 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6830", 2024-09-16T11:44:35.777 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:35.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.777 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.777 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6831", 2024-09-16T11:44:35.777 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:35.777 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.777 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.778 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T11:44:35.778 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T11:44:35.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.778 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T11:44:35.778 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6828", 2024-09-16T11:44:35.778 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:35.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.778 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T11:44:35.778 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.192:6829", 2024-09-16T11:44:35.778 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 33362 2024-09-16T11:44:35.778 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.779 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.779 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.192:6825/33362", 2024-09-16T11:44:35.779 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.192:6827/33362", 2024-09-16T11:44:35.779 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.192:6831/33362", 2024-09-16T11:44:35.779 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.192:6829/33362", 2024-09-16T11:44:35.779 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T11:44:35.779 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T11:44:35.779 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T11:44:35.779 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.779 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.780 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:35.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2024-09-16T11:44:35.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T11:44:35.780 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:35.780 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:35.780 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:35.780 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:35.781 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:35.781 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.581514+0000", 2024-09-16T11:44:35.781 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:35.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T11:44:35.781 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:35.781 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:35.782 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:35.782 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:35.782 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:35.782 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.674271+0000", 2024-09-16T11:44:35.782 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:35.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T11:44:35.782 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:35.782 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:35.782 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:35.783 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:35.783 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:35.783 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:10.545402+0000", 2024-09-16T11:44:35.783 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:35.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.783 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T11:44:35.783 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:35.783 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:35.783 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:35.783 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:35.784 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:35.784 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.733565+0000", 2024-09-16T11:44:35.784 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:35.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.784 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T11:44:35.784 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:35.784 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:35.784 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:35.784 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:35.784 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:35.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.623479+0000", 2024-09-16T11:44:35.785 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:35.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.785 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T11:44:35.785 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:35.785 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:35.785 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:35.785 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:35.785 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:35.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.730066+0000", 2024-09-16T11:44:35.786 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:35.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.786 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T11:44:35.786 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:35.786 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:35.786 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:35.786 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:35.786 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:35.786 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:10.556650+0000", 2024-09-16T11:44:35.786 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:35.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.787 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T11:44:35.787 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T11:44:35.787 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T11:44:35.787 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T11:44:35.787 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138320759226367, 2024-09-16T11:44:35.787 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T11:44:35.787 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T11:43:09.702368+0000", 2024-09-16T11:44:35.787 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T11:44:35.787 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.787 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:35.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2024-09-16T11:44:35.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2024-09-16T11:44:35.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.8", 2024-09-16T11:44:35.788 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T11:44:35.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.788 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 5, 2024-09-16T11:44:35.788 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 1 2024-09-16T11:44:35.788 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.788 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.789 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.14", 2024-09-16T11:44:35.789 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T11:44:35.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:35.789 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T11:44:35.789 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 7 2024-09-16T11:44:35.789 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.789 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:35.789 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.789 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T11:44:35.790 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2024-09-16T11:44:35.790 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2024-09-16T11:44:35.790 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2024-09-16T11:44:35.790 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2024-09-16T11:44:35.790 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2024-09-16T11:44:35.790 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2024-09-16T11:44:35.790 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2024-09-16T11:44:35.790 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2024-09-16T11:44:35.790 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2024-09-16T11:44:35.790 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2024-09-16T11:44:35.790 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2024-09-16T11:44:35.790 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:35.791 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2024-09-16T11:44:35.791 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2024-09-16T11:44:35.791 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2024-09-16T11:44:35.791 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2024-09-16T11:44:35.791 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2024-09-16T11:44:35.791 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2024-09-16T11:44:35.791 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2024-09-16T11:44:35.791 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2024-09-16T11:44:35.791 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2024-09-16T11:44:35.791 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2024-09-16T11:44:35.791 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2024-09-16T11:44:35.791 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:35.792 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:35.792 INFO:teuthology.orchestra.run.smithi070.stdout:5993 writesame 0xd133046 thru 0xd140d20 data_size 0x29(0xdcdb bytes) 2024-09-16T11:44:35.793 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:35.793 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:35.794 INFO:tasks.thrashosds.thrasher:j is {'epoch': 83, 'fsid': '3420bcad-625f-4e15-aaff-fb4236322168', 'created': '2024-09-16T11:43:01.654626+0000', 'modified': '2024-09-16T11:44:34.003349+0000', 'last_up_change': '2024-09-16T11:43:11.689589+0000', 'last_in_change': '2024-09-16T11:43:04.526236+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T11:43:11.406582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T11:43:13.125709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '23', 'last_force_op_resend_prenautilus': '23', 'last_force_op_resend_preluminous': '23', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-09-16T11:43:21.227888+0000', 'flags': 8201, 'flags_names': 'hashpspool,incomplete_clones,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '23', 'last_force_op_resend_prenautilus': '23', 'last_force_op_resend_preluminous': '23', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T11:43:30.105533+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 43, 'snap_epoch': 83, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': '91f052e4-4c8f-4484-92ce-bd1a7f8f3efb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6800', 'nonce': 33355}, {'type': 'v1', 'addr': '172.21.15.70:6801', 'nonce': 33355}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6802', 'nonce': 33355}, {'type': 'v1', 'addr': '172.21.15.70:6803', 'nonce': 33355}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6806', 'nonce': 33355}, {'type': 'v1', 'addr': '172.21.15.70:6807', 'nonce': 33355}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6804', 'nonce': 33355}, {'type': 'v1', 'addr': '172.21.15.70:6805', 'nonce': 33355}]}, 'public_addr': '172.21.15.70:6801/33355', 'cluster_addr': '172.21.15.70:6803/33355', 'heartbeat_back_addr': '172.21.15.70:6807/33355', 'heartbeat_front_addr': '172.21.15.70:6805/33355', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '568b1280-7365-4094-afe8-b0b5e766de6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6816', 'nonce': 33359}, {'type': 'v1', 'addr': '172.21.15.70:6817', 'nonce': 33359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6818', 'nonce': 33359}, {'type': 'v1', 'addr': '172.21.15.70:6819', 'nonce': 33359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6822', 'nonce': 33359}, {'type': 'v1', 'addr': '172.21.15.70:6823', 'nonce': 33359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6820', 'nonce': 33359}, {'type': 'v1', 'addr': '172.21.15.70:6821', 'nonce': 33359}]}, 'public_addr': '172.21.15.70:6817/33359', 'cluster_addr': '172.21.15.70:6819/33359', 'heartbeat_back_addr': '172.21.15.70:6823/33359', 'heartbeat_front_addr': '172.21.15.70:6821/33359', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '328adfa3-c1b4-4745-9759-7327c8cbd499', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6808', 'nonce': 33356}, {'type': 'v1', 'addr': '172.21.15.70:6809', 'nonce': 33356}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6810', 'nonce': 33356}, {'type': 'v1', 'addr': '172.21.15.70:6811', 'nonce': 33356}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6814', 'nonce': 33356}, {'type': 'v1', 'addr': '172.21.15.70:6815', 'nonce': 33356}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6812', 'nonce': 33356}, {'type': 'v1', 'addr': '172.21.15.70:6813', 'nonce': 33356}]}, 'public_addr': '172.21.15.70:6809/33356', 'cluster_addr': '172.21.15.70:6811/33356', 'heartbeat_back_addr': '172.21.15.70:6815/33356', 'heartbeat_front_addr': '172.21.15.70:6813/33356', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '6df66ae7-e835-4d8f-bd27-243d7a9705b5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6824', 'nonce': 33384}, {'type': 'v1', 'addr': '172.21.15.70:6825', 'nonce': 33384}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6826', 'nonce': 33384}, {'type': 'v1', 'addr': '172.21.15.70:6827', 'nonce': 33384}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6830', 'nonce': 33384}, {'type': 'v1', 'addr': '172.21.15.70:6831', 'nonce': 33384}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6828', 'nonce': 33384}, {'type': 'v1', 'addr': '172.21.15.70:6829', 'nonce': 33384}]}, 'public_addr': '172.21.15.70:6825/33384', 'cluster_addr': '172.21.15.70:6827/33384', 'heartbeat_back_addr': '172.21.15.70:6831/33384', 'heartbeat_front_addr': '172.21.15.70:6829/33384', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '29f52d62-d4c3-49f9-992f-23817f5622d2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 33331}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 33331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6805', 'nonce': 33331}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 33331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6813', 'nonce': 33331}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 33331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6809', 'nonce': 33331}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 33331}]}, 'public_addr': '172.21.15.192:6803/33331', 'cluster_addr': '172.21.15.192:6807/33331', 'heartbeat_back_addr': '172.21.15.192:6815/33331', 'heartbeat_front_addr': '172.21.15.192:6811/33331', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0c1a275a-3c75-4e0b-bd35-d34b64d99c0a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 33333}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 33333}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 33333}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 33333}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 33333}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 33333}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 33333}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 33333}]}, 'public_addr': '172.21.15.192:6817/33333', 'cluster_addr': '172.21.15.192:6819/33333', 'heartbeat_back_addr': '172.21.15.192:6823/33333', 'heartbeat_front_addr': '172.21.15.192:6821/33333', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd79f0070-ce58-48e7-a56a-2762ec76a271', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6801', 'nonce': 33332}, {'type': 'v1', 'addr': '172.21.15.192:6802', 'nonce': 33332}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 33332}, {'type': 'v1', 'addr': '172.21.15.192:6806', 'nonce': 33332}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 33332}, {'type': 'v1', 'addr': '172.21.15.192:6814', 'nonce': 33332}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 33332}, {'type': 'v1', 'addr': '172.21.15.192:6810', 'nonce': 33332}]}, 'public_addr': '172.21.15.192:6802/33332', 'cluster_addr': '172.21.15.192:6806/33332', 'heartbeat_back_addr': '172.21.15.192:6814/33332', 'heartbeat_front_addr': '172.21.15.192:6810/33332', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '537fa5df-3d0a-4dd4-8155-20ee94e41c82', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 16, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6824', 'nonce': 33362}, {'type': 'v1', 'addr': '172.21.15.192:6825', 'nonce': 33362}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6826', 'nonce': 33362}, {'type': 'v1', 'addr': '172.21.15.192:6827', 'nonce': 33362}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6830', 'nonce': 33362}, {'type': 'v1', 'addr': '172.21.15.192:6831', 'nonce': 33362}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6828', 'nonce': 33362}, {'type': 'v1', 'addr': '172.21.15.192:6829', 'nonce': 33362}]}, 'public_addr': '172.21.15.192:6825/33362', 'cluster_addr': '172.21.15.192:6827/33362', 'heartbeat_back_addr': '172.21.15.192:6831/33362', 'heartbeat_front_addr': '172.21.15.192:6829/33362', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.581514+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.674271+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:10.545402+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.733565+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.623479+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.730066+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:10.556650+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:43:09.702368+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '4.8', 'mappings': [{'from': 5, 'to': 1}]}, {'pgid': '4.14', 'mappings': [{'from': 1, 'to': 7}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T11:44:35.795 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:44:35.795 INFO:teuthology.orchestra.run.smithi070.stdout:5994 trunc from 0xd140d21 to 0x1a0042d 2024-09-16T11:44:35.796 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:35.797 INFO:teuthology.orchestra.run.smithi070.stdout:5995 trunc from 0x1a0042d to 0x60a63d7 2024-09-16T11:44:35.797 INFO:teuthology.orchestra.run.smithi070.stdout:5996 write 0x1a8aed5 thru 0x1a8ddf7 (0x2f23 bytes) 2024-09-16T11:44:35.797 INFO:teuthology.orchestra.run.smithi070.stdout:5997 read 0x379706b thru 0x379b339 (0x42cf bytes) 2024-09-16T11:44:35.797 INFO:teuthology.orchestra.run.smithi070.stdout:5998 punch from 0x1080ad7 to 0x1081553, (0xa7c bytes) 2024-09-16T11:44:35.797 INFO:teuthology.orchestra.run.smithi070.stdout:6000 trunc from 0x60a63d7 to 0xe002ada 2024-09-16T11:44:35.797 INFO:teuthology.orchestra.run.smithi070.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-16T11:44:35.798 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:35.791+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:35.896 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:35.895+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:35.997 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:35.996+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:36.009 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:36.009 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:36.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:36.025 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:44:36.107 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:36.106+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:36.199 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:36.197+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:36.245 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:36.245 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:36.260 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:36.301 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:36.300+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:36.336 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:36.339 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:44:36.355 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-16T11:44:35.385675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1127509387,"num_objects":1696,"num_object_clones":276,"num_object_copies":3392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1694,"num_whiteouts":940,"num_read":89813,"num_read_kb":409270,"num_write":15784,"num_write_kb":255386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13457,"ondisk_log_size":13457,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2499112,"kb_used_data":41232,"kb_used_omap":13,"kb_used_meta":2457778,"kb_avail":752475608,"statfs":{"total":773094113280,"available":770535022592,"internally_reserved":0,"allocated":42221568,"data_stored":302638460,"data_compressed":12783925,"data_compressed_allocated":27971584,"data_compressed_original":295441230,"omap_allocated":13632,"internal_metadata":2516765376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":74881902,"num_objects":69,"num_object_clones":25,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-42,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":20,"num_read":1424,"num_read_kb":15842,"num_write":463,"num_write_kb":13283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":2194137,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.392911"},"pg_stats":[{"pgid":"4.1f","version":"79'245","reported_seq":2095,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.217169+0000","last_change":"2024-09-16T11:43:33.782279+0000","last_active":"2024-09-16T11:44:30.217169+0000","last_peered":"2024-09-16T11:44:30.217169+0000","last_clean":"2024-09-16T11:44:30.217169+0000","last_became_active":"2024-09-16T11:43:31.779428+0000","last_became_peered":"2024-09-16T11:43:31.779428+0000","last_unstale":"2024-09-16T11:44:30.217169+0000","last_undegraded":"2024-09-16T11:44:30.217169+0000","last_fullsized":"2024-09-16T11:44:30.217169+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:30.437698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090324899999999996,"stat_sum":{"num_bytes":21614199,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":2652,"num_read_kb":10324,"num_write":243,"num_write_kb":6529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"79'349","reported_seq":1572,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.178842+0000","last_change":"2024-09-16T11:43:34.248024+0000","last_active":"2024-09-16T11:44:30.178842+0000","last_peered":"2024-09-16T11:44:30.178842+0000","last_clean":"2024-09-16T11:44:30.178842+0000","last_became_active":"2024-09-16T11:43:31.889706+0000","last_became_peered":"2024-09-16T11:43:31.889706+0000","last_unstale":"2024-09-16T11:44:30.178842+0000","last_undegraded":"2024-09-16T11:44:30.178842+0000","last_fullsized":"2024-09-16T11:44:30.178842+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:25.242116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027516299999999998,"stat_sum":{"num_bytes":30297350,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":28,"num_read":2040,"num_read_kb":10989,"num_write":406,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"81'180","reported_seq":1273,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T11:44:32.154223+0000","last_change":"2024-09-16T11:43:33.777888+0000","last_active":"2024-09-16T11:44:32.154223+0000","last_peered":"2024-09-16T11:44:32.154223+0000","last_clean":"2024-09-16T11:44:32.154223+0000","last_became_active":"2024-09-16T11:43:31.775021+0000","last_became_peered":"2024-09-16T11:43:31.775021+0000","last_unstale":"2024-09-16T11:44:32.154223+0000","last_undegraded":"2024-09-16T11:44:32.154223+0000","last_fullsized":"2024-09-16T11:44:32.154223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:50.045796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075167799999999998,"stat_sum":{"num_bytes":24335775,"num_objects":44,"num_object_clones":9,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":28,"num_read":1089,"num_read_kb":9773,"num_write":185,"num_write_kb":7971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"79'249","reported_seq":1869,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.167770+0000","last_change":"2024-09-16T11:43:33.772949+0000","last_active":"2024-09-16T11:44:30.167770+0000","last_peered":"2024-09-16T11:44:30.167770+0000","last_clean":"2024-09-16T11:44:30.167770+0000","last_became_active":"2024-09-16T11:43:31.773661+0000","last_became_peered":"2024-09-16T11:43:31.773661+0000","last_unstale":"2024-09-16T11:44:30.167770+0000","last_undegraded":"2024-09-16T11:44:30.167770+0000","last_fullsized":"2024-09-16T11:44:30.167770+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:36.062796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023334699999999999,"stat_sum":{"num_bytes":30583531,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":1843,"num_read_kb":13139,"num_write":260,"num_write_kb":6637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"79'323","reported_seq":2195,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.217753+0000","last_change":"2024-09-16T11:43:33.773384+0000","last_active":"2024-09-16T11:44:30.217753+0000","last_peered":"2024-09-16T11:44:30.217753+0000","last_clean":"2024-09-16T11:44:30.217753+0000","last_became_active":"2024-09-16T11:43:31.774523+0000","last_became_peered":"2024-09-16T11:43:31.774523+0000","last_unstale":"2024-09-16T11:44:30.217753+0000","last_undegraded":"2024-09-16T11:44:30.217753+0000","last_fullsized":"2024-09-16T11:44:30.217753+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:15.703792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038485,"stat_sum":{"num_bytes":30590911,"num_objects":53,"num_object_clones":6,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":32,"num_read":2865,"num_read_kb":15182,"num_write":284,"num_write_kb":4200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"79'464","reported_seq":2919,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.365733+0000","last_change":"2024-09-16T11:43:33.774141+0000","last_active":"2024-09-16T11:44:30.365733+0000","last_peered":"2024-09-16T11:44:30.365733+0000","last_clean":"2024-09-16T11:44:30.365733+0000","last_became_active":"2024-09-16T11:43:31.774794+0000","last_became_peered":"2024-09-16T11:43:31.774794+0000","last_unstale":"2024-09-16T11:44:30.365733+0000","last_undegraded":"2024-09-16T11:44:30.365733+0000","last_fullsized":"2024-09-16T11:44:30.365733+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:06:57.981702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010218040000000001,"stat_sum":{"num_bytes":23301388,"num_objects":65,"num_object_clones":10,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":38,"num_read":3654,"num_read_kb":10184,"num_write":592,"num_write_kb":7602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"81'600","reported_seq":1190,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T11:44:32.212658+0000","last_change":"2024-09-16T11:43:33.777212+0000","last_active":"2024-09-16T11:44:32.212658+0000","last_peered":"2024-09-16T11:44:32.212658+0000","last_clean":"2024-09-16T11:44:32.212658+0000","last_became_active":"2024-09-16T11:43:31.769719+0000","last_became_peered":"2024-09-16T11:43:31.769719+0000","last_unstale":"2024-09-16T11:44:32.212658+0000","last_undegraded":"2024-09-16T11:44:32.212658+0000","last_fullsized":"2024-09-16T11:44:32.212658+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:06.911046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035706099999999999,"stat_sum":{"num_bytes":42709170,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":24,"num_read":2401,"num_read_kb":4316,"num_write":823,"num_write_kb":4342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"79'173","reported_seq":1172,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.285179+0000","last_change":"2024-09-16T11:43:33.774063+0000","last_active":"2024-09-16T11:44:30.285179+0000","last_peered":"2024-09-16T11:44:30.285179+0000","last_clean":"2024-09-16T11:44:30.285179+0000","last_became_active":"2024-09-16T11:43:31.776951+0000","last_became_peered":"2024-09-16T11:43:31.776951+0000","last_unstale":"2024-09-16T11:44:30.285179+0000","last_undegraded":"2024-09-16T11:44:30.285179+0000","last_fullsized":"2024-09-16T11:44:30.285179+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:28:39.026621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094366700000000003,"stat_sum":{"num_bytes":27398683,"num_objects":37,"num_object_clones":3,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":22,"num_read":1325,"num_read_kb":6834,"num_write":151,"num_write_kb":5541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"81'415","reported_seq":1802,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T11:44:32.159153+0000","last_change":"2024-09-16T11:43:33.777913+0000","last_active":"2024-09-16T11:44:32.159153+0000","last_peered":"2024-09-16T11:44:32.159153+0000","last_clean":"2024-09-16T11:44:32.159153+0000","last_became_active":"2024-09-16T11:43:31.889807+0000","last_became_peered":"2024-09-16T11:43:31.889807+0000","last_unstale":"2024-09-16T11:44:32.159153+0000","last_undegraded":"2024-09-16T11:44:32.159153+0000","last_fullsized":"2024-09-16T11:44:32.159153+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:30.698410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071799400000000001,"stat_sum":{"num_bytes":63755307,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":30,"num_read":3479,"num_read_kb":13802,"num_write":468,"num_write_kb":8685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"79'564","reported_seq":1835,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.132950+0000","last_change":"2024-09-16T11:43:34.248043+0000","last_active":"2024-09-16T11:44:30.132950+0000","last_peered":"2024-09-16T11:44:30.132950+0000","last_clean":"2024-09-16T11:44:30.132950+0000","last_became_active":"2024-09-16T11:43:31.776111+0000","last_became_peered":"2024-09-16T11:43:31.776111+0000","last_unstale":"2024-09-16T11:44:30.132950+0000","last_undegraded":"2024-09-16T11:44:30.132950+0000","last_fullsized":"2024-09-16T11:44:30.132950+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:21:28.658052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024883199999999999,"stat_sum":{"num_bytes":28151106,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":29,"num_read":3081,"num_read_kb":17612,"num_write":689,"num_write_kb":12425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"80'575","reported_seq":2409,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T11:44:32.040773+0000","last_change":"2024-09-16T11:43:33.785207+0000","last_active":"2024-09-16T11:44:32.040773+0000","last_peered":"2024-09-16T11:44:32.040773+0000","last_clean":"2024-09-16T11:44:32.040773+0000","last_became_active":"2024-09-16T11:43:31.889629+0000","last_became_peered":"2024-09-16T11:43:31.889629+0000","last_unstale":"2024-09-16T11:44:32.040773+0000","last_undegraded":"2024-09-16T11:44:32.040773+0000","last_fullsized":"2024-09-16T11:44:32.040773+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:53.124562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022324199999999999,"stat_sum":{"num_bytes":50477557,"num_objects":70,"num_object_clones":11,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":36,"num_read":3722,"num_read_kb":26276,"num_write":753,"num_write_kb":15270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"81'611","reported_seq":1691,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T11:44:32.398029+0000","last_change":"2024-09-16T11:44:28.991455+0000","last_active":"2024-09-16T11:44:32.398029+0000","last_peered":"2024-09-16T11:44:32.398029+0000","last_clean":"2024-09-16T11:44:32.398029+0000","last_became_active":"2024-09-16T11:44:28.989775+0000","last_became_peered":"2024-09-16T11:44:28.989775+0000","last_unstale":"2024-09-16T11:44:32.398029+0000","last_undegraded":"2024-09-16T11:44:32.398029+0000","last_fullsized":"2024-09-16T11:44:32.398029+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:13:30.919402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085420199999999998,"stat_sum":{"num_bytes":46748842,"num_objects":59,"num_object_clones":11,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":28,"num_read":2843,"num_read_kb":14771,"num_write":882,"num_write_kb":10740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3197941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"79'436","reported_seq":1967,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.200462+0000","last_change":"2024-09-16T11:43:33.773601+0000","last_active":"2024-09-16T11:44:30.200462+0000","last_peered":"2024-09-16T11:44:30.200462+0000","last_clean":"2024-09-16T11:44:30.200462+0000","last_became_active":"2024-09-16T11:43:31.776781+0000","last_became_peered":"2024-09-16T11:43:31.776781+0000","last_unstale":"2024-09-16T11:44:30.200462+0000","last_undegraded":"2024-09-16T11:44:30.200462+0000","last_fullsized":"2024-09-16T11:44:30.200462+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:14.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056674299999999998,"stat_sum":{"num_bytes":37817394,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":33,"num_read":3361,"num_read_kb":14690,"num_write":521,"num_write_kb":10599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"81'261","reported_seq":1634,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T11:44:32.247010+0000","last_change":"2024-09-16T11:43:33.773531+0000","last_active":"2024-09-16T11:44:32.247010+0000","last_peered":"2024-09-16T11:44:32.247010+0000","last_clean":"2024-09-16T11:44:32.247010+0000","last_became_active":"2024-09-16T11:43:31.780164+0000","last_became_peered":"2024-09-16T11:43:31.780164+0000","last_unstale":"2024-09-16T11:44:32.247010+0000","last_undegraded":"2024-09-16T11:44:32.247010+0000","last_fullsized":"2024-09-16T11:44:32.247010+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:56.846891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033807800000000001,"stat_sum":{"num_bytes":21220696,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":1647,"num_read_kb":9784,"num_write":276,"num_write_kb":6053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"81'693","reported_seq":1786,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T11:44:32.247708+0000","last_change":"2024-09-16T11:43:33.777531+0000","last_active":"2024-09-16T11:44:32.247708+0000","last_peered":"2024-09-16T11:44:32.247708+0000","last_clean":"2024-09-16T11:44:32.247708+0000","last_became_active":"2024-09-16T11:43:31.889754+0000","last_became_peered":"2024-09-16T11:43:31.889754+0000","last_unstale":"2024-09-16T11:44:32.247708+0000","last_undegraded":"2024-09-16T11:44:32.247708+0000","last_fullsized":"2024-09-16T11:44:32.247708+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:54.752770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062624299999999996,"stat_sum":{"num_bytes":23936366,"num_objects":69,"num_object_clones":9,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":47,"num_read":3804,"num_read_kb":17359,"num_write":968,"num_write_kb":8873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"81'300","reported_seq":1473,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T11:44:32.191971+0000","last_change":"2024-09-16T11:43:33.777407+0000","last_active":"2024-09-16T11:44:32.191971+0000","last_peered":"2024-09-16T11:44:32.191971+0000","last_clean":"2024-09-16T11:44:32.191971+0000","last_became_active":"2024-09-16T11:43:31.775962+0000","last_became_peered":"2024-09-16T11:43:31.775962+0000","last_unstale":"2024-09-16T11:44:32.191971+0000","last_undegraded":"2024-09-16T11:44:32.191971+0000","last_fullsized":"2024-09-16T11:44:32.191971+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:46.492489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062432399999999999,"stat_sum":{"num_bytes":18452737,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":32,"num_read":2584,"num_read_kb":9067,"num_write":278,"num_write_kb":2023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T11:44:28.985526+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:44:28.985526+0000","last_peered":"2024-09-16T11:44:28.985526+0000","last_clean":"2024-09-16T11:44:28.985526+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:44:28.985526+0000","last_undegraded":"2024-09-16T11:44:28.985526+0000","last_fullsized":"2024-09-16T11:44:28.985526+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"79'288","reported_seq":1343,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.371061+0000","last_change":"2024-09-16T11:43:33.773359+0000","last_active":"2024-09-16T11:44:30.371061+0000","last_peered":"2024-09-16T11:44:30.371061+0000","last_clean":"2024-09-16T11:44:30.371061+0000","last_became_active":"2024-09-16T11:43:31.777591+0000","last_became_peered":"2024-09-16T11:43:31.777591+0000","last_unstale":"2024-09-16T11:44:30.371061+0000","last_undegraded":"2024-09-16T11:44:30.371061+0000","last_fullsized":"2024-09-16T11:44:30.371061+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:00:39.586913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037893300000000003,"stat_sum":{"num_bytes":22893926,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":24,"num_read":2192,"num_read_kb":8156,"num_write":252,"num_write_kb":9480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":75,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T11:44:31.009310+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:44:31.009310+0000","last_peered":"2024-09-16T11:44:31.009310+0000","last_clean":"2024-09-16T11:44:31.009310+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:44:31.009310+0000","last_undegraded":"2024-09-16T11:44:31.009310+0000","last_fullsized":"2024-09-16T11:44:31.009310+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"81'391","reported_seq":2763,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T11:44:32.398265+0000","last_change":"2024-09-16T11:43:33.777644+0000","last_active":"2024-09-16T11:44:32.398265+0000","last_peered":"2024-09-16T11:44:32.398265+0000","last_clean":"2024-09-16T11:44:32.398265+0000","last_became_active":"2024-09-16T11:43:31.892159+0000","last_became_peered":"2024-09-16T11:43:31.892159+0000","last_unstale":"2024-09-16T11:44:32.398265+0000","last_undegraded":"2024-09-16T11:44:32.398265+0000","last_fullsized":"2024-09-16T11:44:32.398265+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:42:14.042476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062271400000000001,"stat_sum":{"num_bytes":48469570,"num_objects":64,"num_object_clones":12,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":33,"num_read":3728,"num_read_kb":18280,"num_write":456,"num_write_kb":8209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":75,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T11:44:31.008385+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:44:31.008385+0000","last_peered":"2024-09-16T11:44:31.008385+0000","last_clean":"2024-09-16T11:44:31.008385+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:44:31.008385+0000","last_undegraded":"2024-09-16T11:44:31.008385+0000","last_fullsized":"2024-09-16T11:44:31.008385+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"79'402","reported_seq":1244,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.268903+0000","last_change":"2024-09-16T11:43:34.247771+0000","last_active":"2024-09-16T11:44:30.268903+0000","last_peered":"2024-09-16T11:44:30.268903+0000","last_clean":"2024-09-16T11:44:30.268903+0000","last_became_active":"2024-09-16T11:43:31.777896+0000","last_became_peered":"2024-09-16T11:43:31.777896+0000","last_unstale":"2024-09-16T11:44:30.268903+0000","last_undegraded":"2024-09-16T11:44:30.268903+0000","last_fullsized":"2024-09-16T11:44:30.268903+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:35.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026844500000000002,"stat_sum":{"num_bytes":46146576,"num_objects":42,"num_object_clones":6,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":24,"num_read":1823,"num_read_kb":8208,"num_write":408,"num_write_kb":6902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":75,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T11:44:31.009063+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:44:31.009063+0000","last_peered":"2024-09-16T11:44:31.009063+0000","last_clean":"2024-09-16T11:44:31.009063+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:44:31.009063+0000","last_undegraded":"2024-09-16T11:44:31.009063+0000","last_fullsized":"2024-09-16T11:44:31.009063+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"80'341","reported_seq":799,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T11:44:32.002270+0000","last_change":"2024-09-16T11:43:33.774367+0000","last_active":"2024-09-16T11:44:32.002270+0000","last_peered":"2024-09-16T11:44:32.002270+0000","last_clean":"2024-09-16T11:44:32.002270+0000","last_became_active":"2024-09-16T11:43:31.778506+0000","last_became_peered":"2024-09-16T11:43:31.778506+0000","last_unstale":"2024-09-16T11:44:32.002270+0000","last_undegraded":"2024-09-16T11:44:32.002270+0000","last_fullsized":"2024-09-16T11:44:32.002270+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:12.232493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010404660000000001,"stat_sum":{"num_bytes":6574951,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":22,"num_read":1393,"num_read_kb":4448,"num_write":469,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":66,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T11:44:28.993755+0000","last_change":"2024-09-16T11:43:22.438212+0000","last_active":"2024-09-16T11:44:28.993755+0000","last_peered":"2024-09-16T11:44:28.993755+0000","last_clean":"2024-09-16T11:44:28.993755+0000","last_became_active":"2024-09-16T11:43:22.437868+0000","last_became_peered":"2024-09-16T11:43:22.437868+0000","last_unstale":"2024-09-16T11:44:28.993755+0000","last_undegraded":"2024-09-16T11:44:28.993755+0000","last_fullsized":"2024-09-16T11:44:28.993755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:35.498933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":77,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T11:44:31.006449+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:44:31.006449+0000","last_peered":"2024-09-16T11:44:31.006449+0000","last_clean":"2024-09-16T11:44:31.006449+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:44:31.006449+0000","last_undegraded":"2024-09-16T11:44:31.006449+0000","last_fullsized":"2024-09-16T11:44:31.006449+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"80'650","reported_seq":2191,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T11:44:31.071946+0000","last_change":"2024-09-16T11:43:33.774349+0000","last_active":"2024-09-16T11:44:31.071946+0000","last_peered":"2024-09-16T11:44:31.071946+0000","last_clean":"2024-09-16T11:44:31.071946+0000","last_became_active":"2024-09-16T11:43:31.777796+0000","last_became_peered":"2024-09-16T11:43:31.777796+0000","last_unstale":"2024-09-16T11:44:31.071946+0000","last_undegraded":"2024-09-16T11:44:31.071946+0000","last_fullsized":"2024-09-16T11:44:31.071946+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:33.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033368800000000001,"stat_sum":{"num_bytes":41535244,"num_objects":51,"num_object_clones":10,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":28,"num_read":3933,"num_read_kb":14971,"num_write":852,"num_write_kb":10037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":66,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T11:44:31.009370+0000","last_change":"2024-09-16T11:43:22.440136+0000","last_active":"2024-09-16T11:44:31.009370+0000","last_peered":"2024-09-16T11:44:31.009370+0000","last_clean":"2024-09-16T11:44:31.009370+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:44:31.009370+0000","last_undegraded":"2024-09-16T11:44:31.009370+0000","last_fullsized":"2024-09-16T11:44:31.009370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:18.461948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":75,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T11:44:28.984973+0000","last_change":"2024-09-16T11:43:16.732834+0000","last_active":"2024-09-16T11:44:28.984973+0000","last_peered":"2024-09-16T11:44:28.984973+0000","last_clean":"2024-09-16T11:44:28.984973+0000","last_became_active":"2024-09-16T11:43:14.724728+0000","last_became_peered":"2024-09-16T11:43:14.724728+0000","last_unstale":"2024-09-16T11:44:28.984973+0000","last_undegraded":"2024-09-16T11:44:28.984973+0000","last_fullsized":"2024-09-16T11:44:28.984973+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:47.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"80'468","reported_seq":3056,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T11:44:32.002256+0000","last_change":"2024-09-16T11:43:33.774126+0000","last_active":"2024-09-16T11:44:32.002256+0000","last_peered":"2024-09-16T11:44:32.002256+0000","last_clean":"2024-09-16T11:44:32.002256+0000","last_became_active":"2024-09-16T11:43:31.777967+0000","last_became_peered":"2024-09-16T11:43:31.777967+0000","last_unstale":"2024-09-16T11:44:32.002256+0000","last_undegraded":"2024-09-16T11:44:32.002256+0000","last_fullsized":"2024-09-16T11:44:32.002256+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:43.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000373488,"stat_sum":{"num_bytes":57426134,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":31,"num_read":3527,"num_read_kb":23304,"num_write":573,"num_write_kb":9604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"56'1","reported_seq":66,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T11:44:28.995697+0000","last_change":"2024-09-16T11:43:22.437965+0000","last_active":"2024-09-16T11:44:28.995697+0000","last_peered":"2024-09-16T11:44:28.995697+0000","last_clean":"2024-09-16T11:44:28.995697+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:44:28.995697+0000","last_undegraded":"2024-09-16T11:44:28.995697+0000","last_fullsized":"2024-09-16T11:44:28.995697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:52.884360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":75,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T11:44:31.008319+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:44:31.008319+0000","last_peered":"2024-09-16T11:44:31.008319+0000","last_clean":"2024-09-16T11:44:31.008319+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:44:31.008319+0000","last_undegraded":"2024-09-16T11:44:31.008319+0000","last_fullsized":"2024-09-16T11:44:31.008319+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"79'287","reported_seq":1315,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.255788+0000","last_change":"2024-09-16T11:43:34.248254+0000","last_active":"2024-09-16T11:44:30.255788+0000","last_peered":"2024-09-16T11:44:30.255788+0000","last_clean":"2024-09-16T11:44:30.255788+0000","last_became_active":"2024-09-16T11:43:31.776645+0000","last_became_peered":"2024-09-16T11:43:31.776645+0000","last_unstale":"2024-09-16T11:44:30.255788+0000","last_undegraded":"2024-09-16T11:44:30.255788+0000","last_fullsized":"2024-09-16T11:44:30.255788+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:53.198323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038670399999999998,"stat_sum":{"num_bytes":63314364,"num_objects":60,"num_object_clones":13,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":31,"num_read":1432,"num_read_kb":10572,"num_write":282,"num_write_kb":10201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"56'1","reported_seq":67,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.994146+0000","last_change":"2024-09-16T11:43:22.587076+0000","last_active":"2024-09-16T11:44:30.994146+0000","last_peered":"2024-09-16T11:44:30.994146+0000","last_clean":"2024-09-16T11:44:30.994146+0000","last_became_active":"2024-09-16T11:43:22.586861+0000","last_became_peered":"2024-09-16T11:43:22.586861+0000","last_unstale":"2024-09-16T11:44:30.994146+0000","last_undegraded":"2024-09-16T11:44:30.994146+0000","last_fullsized":"2024-09-16T11:44:30.994146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:07:45.296752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":76,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T11:44:31.006398+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:44:31.006398+0000","last_peered":"2024-09-16T11:44:31.006398+0000","last_clean":"2024-09-16T11:44:31.006398+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:44:31.006398+0000","last_undegraded":"2024-09-16T11:44:31.006398+0000","last_fullsized":"2024-09-16T11:44:31.006398+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":162,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T11:44:31.008406+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:44:31.008406+0000","last_peered":"2024-09-16T11:44:31.008406+0000","last_clean":"2024-09-16T11:44:31.008406+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:44:31.008406+0000","last_undegraded":"2024-09-16T11:44:31.008406+0000","last_fullsized":"2024-09-16T11:44:31.008406+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"79'434","reported_seq":3028,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.167211+0000","last_change":"2024-09-16T11:43:33.781611+0000","last_active":"2024-09-16T11:44:30.167211+0000","last_peered":"2024-09-16T11:44:30.167211+0000","last_clean":"2024-09-16T11:44:30.167211+0000","last_became_active":"2024-09-16T11:43:31.782071+0000","last_became_peered":"2024-09-16T11:43:31.782071+0000","last_unstale":"2024-09-16T11:44:30.167211+0000","last_undegraded":"2024-09-16T11:44:30.167211+0000","last_fullsized":"2024-09-16T11:44:30.167211+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:15.516215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033637000000000001,"stat_sum":{"num_bytes":66692159,"num_objects":65,"num_object_clones":12,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":29,"num_read":3571,"num_read_kb":24178,"num_write":480,"num_write_kb":7885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"81'566","reported_seq":3469,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T11:44:32.416491+0000","last_change":"2024-09-16T11:44:10.921411+0000","last_active":"2024-09-16T11:44:32.416491+0000","last_peered":"2024-09-16T11:44:32.416491+0000","last_clean":"2024-09-16T11:44:32.416491+0000","last_became_active":"2024-09-16T11:44:10.920131+0000","last_became_peered":"2024-09-16T11:44:10.920131+0000","last_unstale":"2024-09-16T11:44:32.416491+0000","last_undegraded":"2024-09-16T11:44:32.416491+0000","last_fullsized":"2024-09-16T11:44:32.416491+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:55:10.158414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398253,"stat_sum":{"num_bytes":77836158,"num_objects":79,"num_object_clones":13,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":42,"num_read":4138,"num_read_kb":36240,"num_write":644,"num_write_kb":19117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":8960357,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"79'602","reported_seq":2906,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.213295+0000","last_change":"2024-09-16T11:43:33.773950+0000","last_active":"2024-09-16T11:44:30.213295+0000","last_peered":"2024-09-16T11:44:30.213295+0000","last_clean":"2024-09-16T11:44:30.213295+0000","last_became_active":"2024-09-16T11:43:31.775309+0000","last_became_peered":"2024-09-16T11:43:31.775309+0000","last_unstale":"2024-09-16T11:44:30.213295+0000","last_undegraded":"2024-09-16T11:44:30.213295+0000","last_fullsized":"2024-09-16T11:44:30.213295+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:23.106465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086187000000000004,"stat_sum":{"num_bytes":38455436,"num_objects":63,"num_object_clones":9,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":34,"num_read":6467,"num_read_kb":7794,"num_write":588,"num_write_kb":5903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"79'499","reported_seq":1349,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.346307+0000","last_change":"2024-09-16T11:43:33.781852+0000","last_active":"2024-09-16T11:44:30.346307+0000","last_peered":"2024-09-16T11:44:30.346307+0000","last_clean":"2024-09-16T11:44:30.346307+0000","last_became_active":"2024-09-16T11:43:31.779384+0000","last_became_peered":"2024-09-16T11:43:31.779384+0000","last_unstale":"2024-09-16T11:44:30.346307+0000","last_undegraded":"2024-09-16T11:44:30.346307+0000","last_fullsized":"2024-09-16T11:44:30.346307+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:34.933146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029212900000000002,"stat_sum":{"num_bytes":21463345,"num_objects":59,"num_object_clones":7,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":37,"num_read":2197,"num_read_kb":8824,"num_write":563,"num_write_kb":6082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"78'309","reported_seq":1255,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T11:44:29.592876+0000","last_change":"2024-09-16T11:43:34.247758+0000","last_active":"2024-09-16T11:44:29.592876+0000","last_peered":"2024-09-16T11:44:29.592876+0000","last_clean":"2024-09-16T11:44:29.592876+0000","last_became_active":"2024-09-16T11:43:31.775335+0000","last_became_peered":"2024-09-16T11:43:31.775335+0000","last_unstale":"2024-09-16T11:44:29.592876+0000","last_undegraded":"2024-09-16T11:44:29.592876+0000","last_fullsized":"2024-09-16T11:44:29.592876+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:07.521713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026563699999999999,"stat_sum":{"num_bytes":29145232,"num_objects":43,"num_object_clones":4,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":25,"num_read":2072,"num_read_kb":7951,"num_write":364,"num_write_kb":9607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"79'380","reported_seq":1626,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.395888+0000","last_change":"2024-09-16T11:43:33.773570+0000","last_active":"2024-09-16T11:44:30.395888+0000","last_peered":"2024-09-16T11:44:30.395888+0000","last_clean":"2024-09-16T11:44:30.395888+0000","last_became_active":"2024-09-16T11:43:31.774601+0000","last_became_peered":"2024-09-16T11:43:31.774601+0000","last_unstale":"2024-09-16T11:44:30.395888+0000","last_undegraded":"2024-09-16T11:44:30.395888+0000","last_fullsized":"2024-09-16T11:44:30.395888+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:12.584420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055828300000000002,"stat_sum":{"num_bytes":13260014,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":29,"num_read":2849,"num_read_kb":4224,"num_write":355,"num_write_kb":2324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"79'551","reported_seq":1587,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.314514+0000","last_change":"2024-09-16T11:43:33.773852+0000","last_active":"2024-09-16T11:44:30.314514+0000","last_peered":"2024-09-16T11:44:30.314514+0000","last_clean":"2024-09-16T11:44:30.314514+0000","last_became_active":"2024-09-16T11:43:31.772506+0000","last_became_peered":"2024-09-16T11:43:31.772506+0000","last_unstale":"2024-09-16T11:44:30.314514+0000","last_undegraded":"2024-09-16T11:44:30.314514+0000","last_fullsized":"2024-09-16T11:44:30.314514+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:36.407693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076283899999999997,"stat_sum":{"num_bytes":23386657,"num_objects":45,"num_object_clones":5,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":28,"num_read":3104,"num_read_kb":14354,"num_write":760,"num_write_kb":9948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"79'460","reported_seq":1273,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T11:44:30.255422+0000","last_change":"2024-09-16T11:43:33.773770+0000","last_active":"2024-09-16T11:44:30.255422+0000","last_peered":"2024-09-16T11:44:30.255422+0000","last_clean":"2024-09-16T11:44:30.255422+0000","last_became_active":"2024-09-16T11:43:31.779613+0000","last_became_peered":"2024-09-16T11:43:31.779613+0000","last_unstale":"2024-09-16T11:44:30.255422+0000","last_undegraded":"2024-09-16T11:44:30.255422+0000","last_fullsized":"2024-09-16T11:44:30.255422+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:40.995663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000720899,"stat_sum":{"num_bytes":41072151,"num_objects":60,"num_object_clones":11,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":30,"num_read":1960,"num_read_kb":8996,"num_write":558,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"81'314","reported_seq":1514,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T11:44:32.149868+0000","last_change":"2024-09-16T11:43:33.773914+0000","last_active":"2024-09-16T11:44:32.149868+0000","last_peered":"2024-09-16T11:44:32.149868+0000","last_clean":"2024-09-16T11:44:32.149868+0000","last_became_active":"2024-09-16T11:43:31.776132+0000","last_became_peered":"2024-09-16T11:43:31.776132+0000","last_unstale":"2024-09-16T11:44:32.149868+0000","last_undegraded":"2024-09-16T11:44:32.149868+0000","last_fullsized":"2024-09-16T11:44:32.149868+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:53.489198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023678600000000001,"stat_sum":{"num_bytes":7856071,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":28,"num_read":2971,"num_read_kb":4613,"num_write":298,"num_write_kb":5851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1126919000,"num_objects":1690,"num_object_clones":276,"num_object_copies":3380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1690,"num_whiteouts":938,"num_read":89747,"num_read_kb":409215,"num_write":15681,"num_write_kb":253756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35889152,"data_stored":309167405,"data_compressed":13609762,"data_compressed_allocated":28942336,"data_compressed_original":305517390,"omap_allocated":0,"internal_metadata":0},"log_size":13380,"ondisk_log_size":13380,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316420,"kb_used_data":6588,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94055420,"statfs":{"total":96636764160,"available":96312750080,"internally_reserved":0,"allocated":6746112,"data_stored":53828846,"data_compressed":2066201,"data_compressed_allocated":4804608,"data_compressed_original":52861869,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.29799999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":3.2469999999999999,"5min":3.2469999999999999,"15min":3.2469999999999999},"last":0.435},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.42099999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.59399999999999997}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.5640000000000001,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":0.51200000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.499}]},{"osd":5,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.56499999999999995}]},{"osd":6,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.55400000000000005}]}]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314336,"kb_used_data":4500,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94057504,"statfs":{"total":96636764160,"available":96314884096,"internally_reserved":0,"allocated":4608000,"data_stored":32778466,"data_compressed":1594427,"data_compressed_allocated":2977792,"data_compressed_original":31945261,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.373}]},{"osd":1,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38400000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.40999999999999998}]},{"osd":3,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.35499999999999998}]},{"osd":4,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.29199999999999998}]},{"osd":6,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.30299999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.32000000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542164,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312272,"kb_used_data":6600,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94059568,"statfs":{"total":96636764160,"available":96316997632,"internally_reserved":0,"allocated":6758400,"data_stored":49994784,"data_compressed":2323737,"data_compressed_allocated":4718592,"data_compressed_original":48962689,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.30399999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.41899999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.45000000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.61499999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.50600000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.59899999999999998}]},{"osd":7,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.57299999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313376,"kb_used_data":7700,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94058464,"statfs":{"total":96636764160,"available":96315867136,"internally_reserved":0,"allocated":7884800,"data_stored":58304950,"data_compressed":2646570,"data_compressed_allocated":5656576,"data_compressed_original":57180473,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.17799999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.23400000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.25},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.26200000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.27800000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.315}]},{"osd":5,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.432}]},{"osd":6,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56000000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.44900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313676,"kb_used_data":3844,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058164,"statfs":{"total":96636764160,"available":96315559936,"internally_reserved":0,"allocated":3936256,"data_stored":28896811,"data_compressed":873644,"data_compressed_allocated":2375680,"data_compressed_original":28122457,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.307},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.375}]},{"osd":1,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41499999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.49099999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.45900000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.442}]},{"osd":5,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.32800000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.39200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302736,"kb_used_data":5376,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069104,"statfs":{"total":96636764160,"available":96326762496,"internally_reserved":0,"allocated":5505024,"data_stored":38128685,"data_compressed":1587119,"data_compressed_allocated":3481600,"data_compressed_original":37077895,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.33000000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.33900000000000002}]},{"osd":2,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.317},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.42199999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.36699999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.223},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.254}]},{"osd":6,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.379}]},{"osd":7,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.30399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312464,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059376,"statfs":{"total":96636764160,"available":96316801024,"internally_reserved":0,"allocated":2691072,"data_stored":14882149,"data_compressed":558537,"data_compressed_allocated":1343488,"data_compressed_original":14205472,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.218},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.28799999999999998}]},{"osd":1,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.307},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.32700000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.39300000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.497}]},{"osd":5,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.71899999999999997}]},{"osd":6,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.58599999999999997}]},{"osd":7,"last update":"Mon Sep 16 11:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.625}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313832,"kb_used_data":3996,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058008,"statfs":{"total":96636764160,"available":96315400192,"internally_reserved":0,"allocated":4091904,"data_stored":25823769,"data_compressed":1133690,"data_compressed_allocated":2613248,"data_compressed_original":25085114,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.38400000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.40300000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.441}]},{"osd":4,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.496}]},{"osd":5,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.53100000000000003}]},{"osd":6,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.505},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.58199999999999996}]},{"osd":7,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3108864,"data_stored":24785704,"data_compressed":1128534,"data_compressed_allocated":2576384,"data_compressed_original":24511674,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7049216,"data_stored":59582580,"data_compressed":2801041,"data_compressed_allocated":5734400,"data_compressed_original":58900793,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1765376,"data_stored":14434452,"data_compressed":558537,"data_compressed_allocated":1343488,"data_compressed_original":14205472,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6213632,"data_stored":53675712,"data_compressed":2595325,"data_compressed_allocated":5062656,"data_compressed_original":53083265,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4636672,"data_stored":37806672,"data_compressed":1607762,"data_compressed_allocated":3514368,"data_compressed_original":37184391,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":36251008,"data_compressed":1840360,"data_compressed_allocated":3452928,"data_compressed_original":35861037,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3067904,"data_stored":28520541,"data_compressed":894131,"data_compressed_allocated":2400256,"data_compressed_original":28179801,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5890048,"data_stored":54110736,"data_compressed":2184072,"data_compressed_allocated":4857856,"data_compressed_original":53590957,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:44:36.358 INFO:tasks.thrashosds.thrasher:Setting 4.7 to [2, 4, 6, 7] 2024-09-16T11:44:36.358 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.7', '2', '4', '6', '7'] 2024-09-16T11:44:36.358 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 4.7 2 4 6 7 2024-09-16T11:44:36.371 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:36.372 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:36.381 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:36.439 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:36.438+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:36.497 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:36.497 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:36.501 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:36.500+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:36.505 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:36.602 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:36.600+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:36.647 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:36.647 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:36.656 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:36.740 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:36.739+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:36.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:44:36.794 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:36.795 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:36.803 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:36.844 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:36.843+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:36.905 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:36.904+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:36.930 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:36.930 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:36.941 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:36.941 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:36.948 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:36.949 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:36.950 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:36.975 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:36.991 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:36.991 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:37.006 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:37.005+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:37.007 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:37.007 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:37.025 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:37.025 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:37.030 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T11:44:37.030 INFO:teuthology.orchestra.run.smithi070.stderr:set 4.7 pg_upmap_items mapping to [2->4,6->7] 2024-09-16T11:44:37.044 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:37.044 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:37.053 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:37.053 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:37.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:37.053 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:37.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.053 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19943 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2416640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524472975999998, 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027947257999999999, 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19943 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5999999999999999e-07 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526297+0000", 2024-09-16T11:44:37.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8059999999999999e-06 2024-09-16T11:44:37.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526303+0000", 2024-09-16T11:44:37.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6579999999999999e-06 2024-09-16T11:44:37.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526306+0000", 2024-09-16T11:44:37.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3969999999999998e-06 2024-09-16T11:44:37.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554165+0000", 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027858519000000002 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554201+0000", 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5908000000000002e-05 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554242+0000", 2024-09-16T11:44:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1510000000000001e-05 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19944 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2842624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524458080999999, 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028052639000000001, 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19944 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:37.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:44:37.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526312+0000", 2024-09-16T11:44:37.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:44:37.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526317+0000", 2024-09-16T11:44:37.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5910000000000003e-06 2024-09-16T11:44:37.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526321+0000", 2024-09-16T11:44:37.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2339999999999999e-06 2024-09-16T11:44:37.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554263+0000", 2024-09-16T11:44:37.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027942181 2024-09-16T11:44:37.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554307+0000", 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4499999999999997e-05 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554362+0000", 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5078999999999999e-05 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19945 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3162112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:37.062 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524443525999999, 2024-09-16T11:44:37.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028145673999999999, 2024-09-16T11:44:37.063 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.063 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.063 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19945 2024-09-16T11:44:37.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.063 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:37.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:37.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526325+0000", 2024-09-16T11:44:37.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:44:37.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526326+0000", 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.733e-06 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526332+0000", 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526336+0000", 2024-09-16T11:44:37.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8410000000000002e-06 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554389+0000", 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028053576 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554427+0000", 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7299999999999999e-05 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554470+0000", 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3195000000000002e-05 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19946 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3268608~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524427947, 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028232587, 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.067 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.068 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.068 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19946 2024-09-16T11:44:37.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.068 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:37.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:37.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526346+0000", 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6459999999999998e-06 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526352+0000", 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1070000000000001e-06 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526355+0000", 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554493+0000", 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.02813746 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554530+0000", 2024-09-16T11:44:37.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6881999999999997e-05 2024-09-16T11:44:37.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554572+0000", 2024-09-16T11:44:37.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2688999999999997e-05 2024-09-16T11:44:37.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.072 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.072 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.072 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19947 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4014080~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.072 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:37.072 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524408592999997, 2024-09-16T11:44:37.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028315646999999999, 2024-09-16T11:44:37.072 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.072 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.073 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.073 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.073 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.073 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19947 2024-09-16T11:44:37.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.073 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:37.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526360+0000", 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.01e-07 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526361+0000", 2024-09-16T11:44:37.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8300000000000001e-06 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526369+0000", 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526377+0000", 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0909999999999993e-06 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554593+0000", 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028216657999999999 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554632+0000", 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8828000000000002e-05 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554675+0000", 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2710000000000003e-05 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19948 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3588096~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524386194000002, 2024-09-16T11:44:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028394545, 2024-09-16T11:44:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19948 2024-09-16T11:44:37.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:37.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9200000000000001e-07 2024-09-16T11:44:37.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526384+0000", 2024-09-16T11:44:37.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:44:37.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526391+0000", 2024-09-16T11:44:37.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5150000000000003e-06 2024-09-16T11:44:37.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526394+0000", 2024-09-16T11:44:37.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4240000000000002e-06 2024-09-16T11:44:37.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554695+0000", 2024-09-16T11:44:37.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028301218999999999 2024-09-16T11:44:37.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554731+0000", 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6298999999999999e-05 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554776+0000", 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4675999999999997e-05 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19949 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1351680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524369921999998, 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028490081, 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19949 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:37.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:37.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:37.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:44:37.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526400+0000", 2024-09-16T11:44:37.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7099999999999999e-06 2024-09-16T11:44:37.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526406+0000", 2024-09-16T11:44:37.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5140000000000001e-06 2024-09-16T11:44:37.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526409+0000", 2024-09-16T11:44:37.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5360000000000001e-06 2024-09-16T11:44:37.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554801+0000", 2024-09-16T11:44:37.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028392378999999999 2024-09-16T11:44:37.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554843+0000", 2024-09-16T11:44:37.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1056000000000001e-05 2024-09-16T11:44:37.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554888+0000", 2024-09-16T11:44:37.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466999999999997e-05 2024-09-16T11:44:37.086 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.086 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.086 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.087 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19950 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1777664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.087 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:37.087 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524354926000001, 2024-09-16T11:44:37.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028661844999999998, 2024-09-16T11:44:37.087 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.087 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.088 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19950 2024-09-16T11:44:37.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.089 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:37.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:37.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526415+0000", 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.776e-06 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526422+0000", 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6930000000000001e-06 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526425+0000", 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554923+0000", 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028498102000000001 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555000+0000", 2024-09-16T11:44:37.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6619999999999995e-05 2024-09-16T11:44:37.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555075+0000", 2024-09-16T11:44:37.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4931999999999993e-05 2024-09-16T11:44:37.092 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.092 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.093 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.093 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19951 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2097152~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.093 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:37.093 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524339114999997, 2024-09-16T11:44:37.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028755718, 2024-09-16T11:44:37.094 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.094 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.094 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19951 2024-09-16T11:44:37.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.094 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526431+0000", 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526441+0000", 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7790000000000007e-06 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526445+0000", 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.634e-06 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555100+0000", 2024-09-16T11:44:37.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654722000000001 2024-09-16T11:44:37.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555138+0000", 2024-09-16T11:44:37.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7996e-05 2024-09-16T11:44:37.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555184+0000", 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6427999999999997e-05 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19952 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2203648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524318555000001, 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028901157, 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.099 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19952 2024-09-16T11:44:37.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.099 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:37.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:37.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526450+0000", 2024-09-16T11:44:37.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:44:37.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526452+0000", 2024-09-16T11:44:37.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9039999999999999e-06 2024-09-16T11:44:37.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526465+0000", 2024-09-16T11:44:37.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3559e-05 2024-09-16T11:44:37.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526470+0000", 2024-09-16T11:44:37.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-09-16T11:44:37.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555217+0000", 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028747287999999999 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555272+0000", 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4885000000000001e-05 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555350+0000", 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8428000000000002e-05 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.102 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.103 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19953 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2523136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.103 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:37.103 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524293962999998, 2024-09-16T11:44:37.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029004309999999998, 2024-09-16T11:44:37.103 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.103 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.104 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19953 2024-09-16T11:44:37.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.104 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:37.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:37.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:37.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-16T11:44:37.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526476+0000", 2024-09-16T11:44:37.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9810000000000002e-06 2024-09-16T11:44:37.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526482+0000", 2024-09-16T11:44:37.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:37.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526485+0000", 2024-09-16T11:44:37.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2760000000000001e-06 2024-09-16T11:44:37.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555383+0000", 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028898113 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555433+0000", 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9753999999999998e-05 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555478+0000", 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5006999999999998e-05 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.107 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.108 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19954 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2949120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.108 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:37.108 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524278965999997, 2024-09-16T11:44:37.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029165054999999999, 2024-09-16T11:44:37.108 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.108 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19954 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:37.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:37.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1799999999999995e-07 2024-09-16T11:44:37.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526491+0000", 2024-09-16T11:44:37.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7320000000000001e-06 2024-09-16T11:44:37.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526497+0000", 2024-09-16T11:44:37.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6869999999999999e-06 2024-09-16T11:44:37.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526500+0000", 2024-09-16T11:44:37.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:44:37.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555513+0000", 2024-09-16T11:44:37.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029012626 2024-09-16T11:44:37.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555576+0000", 2024-09-16T11:44:37.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3288999999999997e-05 2024-09-16T11:44:37.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555654+0000", 2024-09-16T11:44:37.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7861999999999994e-05 2024-09-16T11:44:37.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.113 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.113 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19955 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3375104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.113 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:37.113 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524259663999999, 2024-09-16T11:44:37.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029241979000000001, 2024-09-16T11:44:37.113 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.113 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.114 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19955 2024-09-16T11:44:37.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.114 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:37.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526509+0000", 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526510+0000", 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.844e-06 2024-09-16T11:44:37.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526516+0000", 2024-09-16T11:44:37.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9039999999999997e-06 2024-09-16T11:44:37.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526520+0000", 2024-09-16T11:44:37.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3639999999999999e-06 2024-09-16T11:44:37.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555669+0000", 2024-09-16T11:44:37.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029149628 2024-09-16T11:44:37.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555706+0000", 2024-09-16T11:44:37.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7017000000000002e-05 2024-09-16T11:44:37.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555750+0000", 2024-09-16T11:44:37.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3856999999999997e-05 2024-09-16T11:44:37.118 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.118 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.118 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.118 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19956 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3694592~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.118 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524244709000001, 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029398799999999999, 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19956 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526524+0000", 2024-09-16T11:44:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:44:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526525+0000", 2024-09-16T11:44:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-16T11:44:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526531+0000", 2024-09-16T11:44:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6010000000000002e-06 2024-09-16T11:44:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526535+0000", 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.597e-06 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555785+0000", 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029250235999999999 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555845+0000", 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0393000000000003e-05 2024-09-16T11:44:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555922+0000", 2024-09-16T11:44:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6710000000000002e-05 2024-09-16T11:44:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19957 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4120576~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.524229556000002, 2024-09-16T11:44:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029481157000000001, 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19957 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526539+0000", 2024-09-16T11:44:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526540+0000", 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.759e-06 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526546+0000", 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1079999999999998e-06 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526550+0000", 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.269e-06 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555939+0000", 2024-09-16T11:44:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029389444000000001 2024-09-16T11:44:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555977+0000", 2024-09-16T11:44:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7487999999999998e-05 2024-09-16T11:44:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.556019+0000", 2024-09-16T11:44:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2756999999999997e-05 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:28959 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.041373143999998, 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031145285000000002, 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 28959 2024-09-16T11:44:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009396+0000", 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.296e-06 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030864+0000", 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.021468451 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030892+0000", 2024-09-16T11:44:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7555999999999998e-05 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030902+0000", 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9550000000000004e-06 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030979+0000", 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7137000000000004e-05 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031051+0000", 2024-09-16T11:44:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1475000000000003e-05 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031389+0000", 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033827100000000001 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.039264+0000", 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078751700000000008 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:37.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040484+0000", 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012199579999999999 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040517+0000", 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2960999999999998e-05 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040540+0000", 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3054999999999999e-05 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30605 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 3915776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.807066884999998, 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938655999999999, 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30605 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243704+0000", 2024-09-16T11:44:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2199999999999999e-06 2024-09-16T11:44:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243710+0000", 2024-09-16T11:44:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6340000000000003e-06 2024-09-16T11:44:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243714+0000", 2024-09-16T11:44:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5030000000000002e-06 2024-09-16T11:44:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271450+0000", 2024-09-16T11:44:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027736035999999999 2024-09-16T11:44:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271536+0000", 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6521999999999996e-05 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271640+0000", 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103341 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.138 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30607 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4046848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.807043729999997, 2024-09-16T11:44:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028130446, 2024-09-16T11:44:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30607 2024-09-16T11:44:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243725+0000", 2024-09-16T11:44:37.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2399999999999998e-07 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243727+0000", 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3379999999999999e-06 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243733+0000", 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4280000000000001e-06 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243737+0000", 2024-09-16T11:44:37.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8990000000000003e-06 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271676+0000", 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938481000000001 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271757+0000", 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1396999999999993e-05 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271855+0000", 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7379000000000004e-05 2024-09-16T11:44:37.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30608 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.807026878999999, 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028323879, 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30608 2024-09-16T11:44:37.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:44:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243743+0000", 2024-09-16T11:44:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.762e-06 2024-09-16T11:44:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243749+0000", 2024-09-16T11:44:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5670000000000002e-06 2024-09-16T11:44:37.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243758+0000", 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8309999999999996e-06 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271879+0000", 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028120501999999999 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271965+0000", 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5755999999999997e-05 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.272065+0000", 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000100101 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:56045 4.15 4:ab2ce1b7:::rbd_data.11c1b01b799a.0000000000000020:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 7.8717436279999999, 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032248165000000002, 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:37.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 56045 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179026+0000", 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.556e-06 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179032+0000", 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5379999999999999e-06 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179045+0000", 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.277e-05 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179067+0000", 2024-09-16T11:44:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.243e-05 2024-09-16T11:44:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179120+0000", 2024-09-16T11:44:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2726000000000003e-05 2024-09-16T11:44:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179279+0000", 2024-09-16T11:44:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015865899999999999 2024-09-16T11:44:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.185163+0000", 2024-09-16T11:44:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0058837639999999997 2024-09-16T11:44:37.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211197+0000", 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026034130999999999 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211206+0000", 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2539999999999998e-06 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211245+0000", 2024-09-16T11:44:37.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8924999999999997e-05 2024-09-16T11:44:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211272+0000", 2024-09-16T11:44:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7412000000000001e-05 2024-09-16T11:44:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.153 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:37.154 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:37.155 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:37.156 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:37.153+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:37.156 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:37.256 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:37.254+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:37.300 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:37.300 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:37.300 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:37.300 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:37.308 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:37.357 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:37.354+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:37.446 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:37.446 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:37.446 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:37.446 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:37.446 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:37.454 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:37.458 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:37.456+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:37.554 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:37.554 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:37.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:37.554 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:37.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.555 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19282 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.555 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:37.555 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 42.077997201999999, 2024-09-16T11:44:37.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521151000000002, 2024-09-16T11:44:37.555 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.555 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.555 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19282 2024-09-16T11:44:37.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.556 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:37.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:37.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472838+0000", 2024-09-16T11:44:37.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:44:37.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472848+0000", 2024-09-16T11:44:37.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9799999999999993e-06 2024-09-16T11:44:37.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472858+0000", 2024-09-16T11:44:37.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9429999999999995e-06 2024-09-16T11:44:37.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472862+0000", 2024-09-16T11:44:37.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4460000000000003e-06 2024-09-16T11:44:37.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501277+0000", 2024-09-16T11:44:37.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028414900999999999 2024-09-16T11:44:37.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501322+0000", 2024-09-16T11:44:37.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4880999999999999e-05 2024-09-16T11:44:37.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501358+0000", 2024-09-16T11:44:37.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6245000000000003e-05 2024-09-16T11:44:37.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.559 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.560 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19286 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7979008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.560 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:37.560 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 42.077624270000001, 2024-09-16T11:44:37.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521860999999999, 2024-09-16T11:44:37.560 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.560 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.560 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19286 2024-09-16T11:44:37.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.561 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:37.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:37.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473211+0000", 2024-09-16T11:44:37.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:44:37.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473214+0000", 2024-09-16T11:44:37.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1080000000000001e-06 2024-09-16T11:44:37.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473226+0000", 2024-09-16T11:44:37.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2441e-05 2024-09-16T11:44:37.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473234+0000", 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729999999999998e-06 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501592+0000", 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028358491999999999 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501671+0000", 2024-09-16T11:44:37.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8846999999999999e-05 2024-09-16T11:44:37.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501732+0000", 2024-09-16T11:44:37.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1118000000000007e-05 2024-09-16T11:44:37.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.564 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.565 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20619 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7610368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.565 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:37.565 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.990275101999998, 2024-09-16T11:44:37.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028500152000000001, 2024-09-16T11:44:37.565 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.566 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.566 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20619 2024-09-16T11:44:37.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.567 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:37.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:37.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560560+0000", 2024-09-16T11:44:37.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9499999999999998e-07 2024-09-16T11:44:37.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560561+0000", 2024-09-16T11:44:37.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.071e-06 2024-09-16T11:44:37.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560565+0000", 2024-09-16T11:44:37.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:44:37.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560567+0000", 2024-09-16T11:44:37.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:44:37.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588949+0000", 2024-09-16T11:44:37.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381878999999999 2024-09-16T11:44:37.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588989+0000", 2024-09-16T11:44:37.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0649000000000003e-05 2024-09-16T11:44:37.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589059+0000", 2024-09-16T11:44:37.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0239000000000003e-05 2024-09-16T11:44:37.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.574 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.575 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20620 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7716864~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:37.575 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:37.575 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.990265702000002, 2024-09-16T11:44:37.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028627621999999998, 2024-09-16T11:44:37.575 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.576 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.576 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.576 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.576 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.576 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20620 2024-09-16T11:44:37.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.576 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:37.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:37.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562377+0000", 2024-09-16T11:44:37.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018081729999999999 2024-09-16T11:44:37.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562380+0000", 2024-09-16T11:44:37.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0469999999999998e-06 2024-09-16T11:44:37.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562386+0000", 2024-09-16T11:44:37.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2040000000000002e-06 2024-09-16T11:44:37.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562390+0000", 2024-09-16T11:44:37.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:44:37.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589092+0000", 2024-09-16T11:44:37.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026701798999999998 2024-09-16T11:44:37.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589139+0000", 2024-09-16T11:44:37.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7320000000000001e-05 2024-09-16T11:44:37.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589196+0000", 2024-09-16T11:44:37.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7201999999999999e-05 2024-09-16T11:44:37.584 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.584 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.584 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.584 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41954 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.584 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:37.585 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136559911999999, 2024-09-16T11:44:37.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028594441000000002, 2024-09-16T11:44:37.585 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.585 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.585 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41954 2024-09-16T11:44:37.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.586 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:37.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:37.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414275+0000", 2024-09-16T11:44:37.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:44:37.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414277+0000", 2024-09-16T11:44:37.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:37.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414282+0000", 2024-09-16T11:44:37.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7559999999999996e-06 2024-09-16T11:44:37.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414286+0000", 2024-09-16T11:44:37.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:44:37.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442778+0000", 2024-09-16T11:44:37.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028492237 2024-09-16T11:44:37.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442816+0000", 2024-09-16T11:44:37.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8198000000000003e-05 2024-09-16T11:44:37.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442869+0000", 2024-09-16T11:44:37.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2589000000000001e-05 2024-09-16T11:44:37.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.594 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.595 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41955 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.595 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:37.595 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136544854, 2024-09-16T11:44:37.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028687865, 2024-09-16T11:44:37.595 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.595 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.595 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41955 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:37.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:44:37.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414292+0000", 2024-09-16T11:44:37.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-09-16T11:44:37.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414299+0000", 2024-09-16T11:44:37.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4279999999999999e-06 2024-09-16T11:44:37.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414303+0000", 2024-09-16T11:44:37.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:44:37.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442892+0000", 2024-09-16T11:44:37.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028589564000000001 2024-09-16T11:44:37.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442930+0000", 2024-09-16T11:44:37.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330999999999999e-05 2024-09-16T11:44:37.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442977+0000", 2024-09-16T11:44:37.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7874000000000003e-05 2024-09-16T11:44:37.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.600 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.600 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41956 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.600 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:37.600 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136509536999998, 2024-09-16T11:44:37.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028766896, 2024-09-16T11:44:37.600 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.600 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.601 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41956 2024-09-16T11:44:37.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.601 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:37.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:37.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414326+0000", 2024-09-16T11:44:37.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9900000000000009e-07 2024-09-16T11:44:37.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414330+0000", 2024-09-16T11:44:37.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2339999999999996e-06 2024-09-16T11:44:37.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414341+0000", 2024-09-16T11:44:37.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1358999999999999e-05 2024-09-16T11:44:37.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414347+0000", 2024-09-16T11:44:37.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8100000000000003e-06 2024-09-16T11:44:37.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443002+0000", 2024-09-16T11:44:37.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654331000000002 2024-09-16T11:44:37.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443040+0000", 2024-09-16T11:44:37.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8262000000000001e-05 2024-09-16T11:44:37.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443092+0000", 2024-09-16T11:44:37.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1901000000000003e-05 2024-09-16T11:44:37.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.605 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.606 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41957 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.606 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:37.606 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136482118, 2024-09-16T11:44:37.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028848318000000001, 2024-09-16T11:44:37.606 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.606 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.607 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41957 2024-09-16T11:44:37.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.607 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:37.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:37.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414353+0000", 2024-09-16T11:44:37.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:37.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414355+0000", 2024-09-16T11:44:37.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3489999999999999e-06 2024-09-16T11:44:37.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414364+0000", 2024-09-16T11:44:37.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1449999999999997e-06 2024-09-16T11:44:37.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414368+0000", 2024-09-16T11:44:37.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8940000000000003e-06 2024-09-16T11:44:37.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443115+0000", 2024-09-16T11:44:37.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028746532000000002 2024-09-16T11:44:37.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443152+0000", 2024-09-16T11:44:37.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6814999999999999e-05 2024-09-16T11:44:37.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443201+0000", 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9069000000000001e-05 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41958 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136458336, 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028939856, 2024-09-16T11:44:37.611 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.612 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.612 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41958 2024-09-16T11:44:37.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.612 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:37.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414377+0000", 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4099999999999999e-07 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414379+0000", 2024-09-16T11:44:37.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.418e-06 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414385+0000", 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999999e-06 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414389+0000", 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7740000000000002e-06 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443223+0000", 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028833919 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443263+0000", 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0148999999999998e-05 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443316+0000", 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2636000000000002e-05 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.617 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41959 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.617 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:37.617 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136441411, 2024-09-16T11:44:37.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029040309, 2024-09-16T11:44:37.617 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.617 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.617 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41959 2024-09-16T11:44:37.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.618 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:37.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:37.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:37.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:44:37.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414395+0000", 2024-09-16T11:44:37.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7579999999999999e-06 2024-09-16T11:44:37.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414401+0000", 2024-09-16T11:44:37.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5330000000000002e-06 2024-09-16T11:44:37.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414404+0000", 2024-09-16T11:44:37.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.529e-06 2024-09-16T11:44:37.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443339+0000", 2024-09-16T11:44:37.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028934933999999999 2024-09-16T11:44:37.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443383+0000", 2024-09-16T11:44:37.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3832000000000003e-05 2024-09-16T11:44:37.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443433+0000", 2024-09-16T11:44:37.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0309000000000002e-05 2024-09-16T11:44:37.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.622 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.623 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41960 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.623 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:37.623 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136426577000002, 2024-09-16T11:44:37.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029138134, 2024-09-16T11:44:37.623 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.623 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.623 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41960 2024-09-16T11:44:37.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.624 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:37.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:37.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:37.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-07 2024-09-16T11:44:37.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414410+0000", 2024-09-16T11:44:37.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:44:37.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414417+0000", 2024-09-16T11:44:37.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:37.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414420+0000", 2024-09-16T11:44:37.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:44:37.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443457+0000", 2024-09-16T11:44:37.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029037065000000001 2024-09-16T11:44:37.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443495+0000", 2024-09-16T11:44:37.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7555000000000002e-05 2024-09-16T11:44:37.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443546+0000", 2024-09-16T11:44:37.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1422000000000003e-05 2024-09-16T11:44:37.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.631 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.632 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41961 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.632 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:37.632 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136410596000001, 2024-09-16T11:44:37.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029230973, 2024-09-16T11:44:37.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.633 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.633 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41961 2024-09-16T11:44:37.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.633 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:37.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:37.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:37.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4499999999999998e-07 2024-09-16T11:44:37.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414432+0000", 2024-09-16T11:44:37.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.413e-06 2024-09-16T11:44:37.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414439+0000", 2024-09-16T11:44:37.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4769999999999997e-06 2024-09-16T11:44:37.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414443+0000", 2024-09-16T11:44:37.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8859999999999997e-06 2024-09-16T11:44:37.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443569+0000", 2024-09-16T11:44:37.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029126124 2024-09-16T11:44:37.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443606+0000", 2024-09-16T11:44:37.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6588999999999998e-05 2024-09-16T11:44:37.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443655+0000", 2024-09-16T11:44:37.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9138999999999998e-05 2024-09-16T11:44:37.638 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.638 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.639 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41962 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.639 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:37.639 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136387629000001, 2024-09-16T11:44:37.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029324505000000001, 2024-09-16T11:44:37.639 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.639 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41962 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:37.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414449+0000", 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0279999999999999e-06 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414456+0000", 2024-09-16T11:44:37.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3339999999999998e-06 2024-09-16T11:44:37.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414459+0000", 2024-09-16T11:44:37.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.382e-06 2024-09-16T11:44:37.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443678+0000", 2024-09-16T11:44:37.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029218958999999999 2024-09-16T11:44:37.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443719+0000", 2024-09-16T11:44:37.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0722000000000001e-05 2024-09-16T11:44:37.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443771+0000", 2024-09-16T11:44:37.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2661000000000003e-05 2024-09-16T11:44:37.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.643 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.644 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41963 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.644 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:37.644 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136371726, 2024-09-16T11:44:37.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029416557999999999, 2024-09-16T11:44:37.644 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.644 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.644 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41963 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414465+0000", 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414470+0000", 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4750000000000001e-06 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414474+0000", 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3720000000000001e-06 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443795+0000", 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029320821 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443830+0000", 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5463000000000002e-05 2024-09-16T11:44:37.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443879+0000", 2024-09-16T11:44:37.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9147000000000002e-05 2024-09-16T11:44:37.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.648 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.648 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41964 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.648 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:37.648 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136356838000001, 2024-09-16T11:44:37.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029521202999999999, 2024-09-16T11:44:37.649 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.649 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.649 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41964 2024-09-16T11:44:37.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.649 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414480+0000", 2024-09-16T11:44:37.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:44:37.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414486+0000", 2024-09-16T11:44:37.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:37.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414489+0000", 2024-09-16T11:44:37.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2770000000000001e-06 2024-09-16T11:44:37.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443903+0000", 2024-09-16T11:44:37.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029414109000000001 2024-09-16T11:44:37.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443944+0000", 2024-09-16T11:44:37.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0992999999999999e-05 2024-09-16T11:44:37.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443999+0000", 2024-09-16T11:44:37.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4632000000000001e-05 2024-09-16T11:44:37.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.654 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.654 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41965 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.654 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:37.654 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136341875999999, 2024-09-16T11:44:37.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029611293, 2024-09-16T11:44:37.654 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.654 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41965 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:37.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:37.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414495+0000", 2024-09-16T11:44:37.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.88e-06 2024-09-16T11:44:37.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414501+0000", 2024-09-16T11:44:37.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2489999999999998e-06 2024-09-16T11:44:37.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414504+0000", 2024-09-16T11:44:37.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3139999999999999e-06 2024-09-16T11:44:37.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444023+0000", 2024-09-16T11:44:37.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029518564000000001 2024-09-16T11:44:37.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444058+0000", 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5196e-05 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444104+0000", 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5732999999999997e-05 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.659 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41966 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.659 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:37.659 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136325670000002, 2024-09-16T11:44:37.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029707678000000001, 2024-09-16T11:44:37.659 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.659 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.659 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.659 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.659 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.659 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41966 2024-09-16T11:44:37.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414514+0000", 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414518+0000", 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3839999999999999e-06 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414521+0000", 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0400000000000001e-06 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444128+0000", 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029606612000000001 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444167+0000", 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9418000000000001e-05 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444216+0000", 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9354999999999998e-05 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41967 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136309861000001, 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029796916, 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.663 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41967 2024-09-16T11:44:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:37.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:37.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6699999999999999e-07 2024-09-16T11:44:37.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414527+0000", 2024-09-16T11:44:37.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7060000000000001e-06 2024-09-16T11:44:37.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414533+0000", 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2140000000000001e-06 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414536+0000", 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444239+0000", 2024-09-16T11:44:37.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029702698999999999 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444273+0000", 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4229e-05 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444321+0000", 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8477999999999999e-05 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41968 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136294964000001, 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029902765000000001, 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41968 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:37.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:37.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414540+0000", 2024-09-16T11:44:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414542+0000", 2024-09-16T11:44:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:44:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414547+0000", 2024-09-16T11:44:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4639999999999997e-06 2024-09-16T11:44:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414551+0000", 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6279999999999998e-06 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444346+0000", 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029794885 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444389+0000", 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3442000000000002e-05 2024-09-16T11:44:37.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444442+0000", 2024-09-16T11:44:37.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3173000000000001e-05 2024-09-16T11:44:37.672 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.672 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.672 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.672 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41969 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:37.672 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:37.672 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.136280052, 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029997612999999999, 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41969 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414555+0000", 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414556+0000", 2024-09-16T11:44:37.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.725e-06 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414563+0000", 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2530000000000001e-06 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414566+0000", 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4429999999999999e-06 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:37.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444463+0000", 2024-09-16T11:44:37.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029897271 2024-09-16T11:44:37.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:37.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444503+0000", 2024-09-16T11:44:37.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9782e-05 2024-09-16T11:44:37.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:37.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:37.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:37.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444552+0000", 2024-09-16T11:44:37.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8789000000000003e-05 2024-09-16T11:44:37.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.677 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:37.677 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:37.677 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:37.678 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:37.679 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:37.676+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:37.777 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:37.778 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:37.778 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:37.778 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:37.779 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:37.777+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:37.786 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:37.879 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:37.878+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:37.884 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:37.884 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:37.884 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:37.884 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:37.884 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:37.893 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:37.981 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:37.980+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:38.032 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:38.032 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:38.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:38.032 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:38.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.032 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:38.032 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:38.032 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.109300547999993, 2024-09-16T11:44:38.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:44:38.033 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.033 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.033 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:38.033 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:38.033 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:38.033 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:44:38.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.033 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:38.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.109254856000007, 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:38.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.043 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.109197925999993, 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.044 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:38.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:38.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:44:38.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.109172557999997, 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:44:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:44:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:44:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:44:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:44:38.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:38.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:38.058 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.109120590000003, 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:44:38.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:44:38.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:44:38.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:44:38.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:44:38.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.109039480000007, 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.064 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:38.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:38.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:38.065 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:44:38.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.065 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:38.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:44:38.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.109013501999996, 2024-09-16T11:44:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:44:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:38.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:44:38.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:38.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.691494876999997, 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.076 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:38.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:38.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:44:38.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:44:38.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:44:38.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:44:38.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.081 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.691472946000005, 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:44:38.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:44:38.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:44:38.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:44:38.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:38.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:44:38.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.691396057999995, 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.087 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:38.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:38.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:38.088 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:44:38.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.088 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:38.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:44:38.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:44:38.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:38.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:44:38.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:44:38.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:38.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:44:38.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:44:38.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:38.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:44:38.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:44:38.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:44:38.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:44:38.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:44:38.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:44:38.093 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.093 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.093 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.093 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28187 4.17 e52/28 4:ee916bd9:::rbd_data.11c1ce740b46.0000000000000000:head v 52'195, mlcod=52'193)", 2024-09-16T11:44:38.093 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:38.094 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.519976671999999, 2024-09-16T11:44:38.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075854429999999999, 2024-09-16T11:44:38.094 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.094 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.094 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:38.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509349+0000", 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2300000000000001e-06 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513315+0000", 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039660590000000001 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513328+0000", 2024-09-16T11:44:38.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.308e-05 2024-09-16T11:44:38.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513334+0000", 2024-09-16T11:44:38.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:38.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513385+0000", 2024-09-16T11:44:38.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1347000000000002e-05 2024-09-16T11:44:38.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513407+0000", 2024-09-16T11:44:38.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1852999999999999e-05 2024-09-16T11:44:38.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516882+0000", 2024-09-16T11:44:38.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034749690000000001 2024-09-16T11:44:38.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516933+0000", 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0952999999999998e-05 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28959 4.8 e53/28 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head v 53'408, mlcod=53'406)", 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.997827616000002, 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088740420000000004, 2024-09-16T11:44:38.098 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.099 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.099 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:38.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:38.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031498+0000", 2024-09-16T11:44:38.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.652e-06 2024-09-16T11:44:38.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033515+0000", 2024-09-16T11:44:38.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020165930000000001 2024-09-16T11:44:38.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033539+0000", 2024-09-16T11:44:38.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4295999999999999e-05 2024-09-16T11:44:38.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033548+0000", 2024-09-16T11:44:38.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1239999999999994e-06 2024-09-16T11:44:38.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033610+0000", 2024-09-16T11:44:38.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1371999999999994e-05 2024-09-16T11:44:38.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033637+0000", 2024-09-16T11:44:38.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7098999999999999e-05 2024-09-16T11:44:38.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040341+0000", 2024-09-16T11:44:38.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067045819999999997 2024-09-16T11:44:38.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040371+0000", 2024-09-16T11:44:38.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9323999999999999e-05 2024-09-16T11:44:38.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.103 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.103 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32303 4.8 e53/28 4:1436ac82:::rbd_data.11c1cab081b3.0000000000000022:head v 53'417, mlcod=53'415)", 2024-09-16T11:44:38.103 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:38.103 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.527401642000001, 2024-09-16T11:44:38.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085565160000000001, 2024-09-16T11:44:38.104 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.104 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.104 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:38.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:38.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501930+0000", 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7649999999999996e-06 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501946+0000", 2024-09-16T11:44:38.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5301000000000001e-05 2024-09-16T11:44:38.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501952+0000", 2024-09-16T11:44:38.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:44:38.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502100+0000", 2024-09-16T11:44:38.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014847600000000001 2024-09-16T11:44:38.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502120+0000", 2024-09-16T11:44:38.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9303000000000001e-05 2024-09-16T11:44:38.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510436+0000", 2024-09-16T11:44:38.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083166420000000008 2024-09-16T11:44:38.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510479+0000", 2024-09-16T11:44:38.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2979000000000003e-05 2024-09-16T11:44:38.107 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.108 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.108 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32306 4.1e e53/28 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head v 53'124, mlcod=53'122)", 2024-09-16T11:44:38.108 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:38.108 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.527335647999998, 2024-09-16T11:44:38.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084990900000000008, 2024-09-16T11:44:38.108 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.108 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.108 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:38.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:38.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:44:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501994+0000", 2024-09-16T11:44:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:44:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502002+0000", 2024-09-16T11:44:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3839999999999997e-06 2024-09-16T11:44:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502007+0000", 2024-09-16T11:44:38.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6290000000000001e-06 2024-09-16T11:44:38.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502454+0000", 2024-09-16T11:44:38.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00044736800000000002 2024-09-16T11:44:38.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502471+0000", 2024-09-16T11:44:38.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6648999999999999e-05 2024-09-16T11:44:38.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510455+0000", 2024-09-16T11:44:38.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0079841909999999999 2024-09-16T11:44:38.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510488+0000", 2024-09-16T11:44:38.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2948999999999999e-05 2024-09-16T11:44:38.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.113 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.113 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32307 4.8 e53/28 4:11c1c37d:::rbd_data.11c1cab081b3.0000000000000026:head v 53'418, mlcod=53'415)", 2024-09-16T11:44:38.113 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:38.113 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.526580312999997, 2024-09-16T11:44:38.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00776212, 2024-09-16T11:44:38.113 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.113 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.114 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:38.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:38.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:38.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:38.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502750+0000", 2024-09-16T11:44:38.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:44:38.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502760+0000", 2024-09-16T11:44:38.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:44:38.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502767+0000", 2024-09-16T11:44:38.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4219999999999997e-06 2024-09-16T11:44:38.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502802+0000", 2024-09-16T11:44:38.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5089999999999998e-05 2024-09-16T11:44:38.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502816+0000", 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4092e-05 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510483+0000", 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00766704 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510506+0000", 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3195999999999999e-05 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33583 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'416, mlcod=53'411)", 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.818621903999997, 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078132459999999994, 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:38.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210703+0000", 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210706+0000", 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-06 2024-09-16T11:44:38.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210712+0000", 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210715+0000", 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210736+0000", 2024-09-16T11:44:38.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0293999999999999e-05 2024-09-16T11:44:38.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210746+0000", 2024-09-16T11:44:38.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0178e-05 2024-09-16T11:44:38.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218477+0000", 2024-09-16T11:44:38.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007731501 2024-09-16T11:44:38.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218516+0000", 2024-09-16T11:44:38.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8148000000000001e-05 2024-09-16T11:44:38.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.122 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.122 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33590 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'419, mlcod=53'415)", 2024-09-16T11:44:38.122 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:38.122 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.807563762999997, 2024-09-16T11:44:38.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082499879999999998, 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221761+0000", 2024-09-16T11:44:38.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-16T11:44:38.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221765+0000", 2024-09-16T11:44:38.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1590000000000003e-06 2024-09-16T11:44:38.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221774+0000", 2024-09-16T11:44:38.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7490000000000004e-06 2024-09-16T11:44:38.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221778+0000", 2024-09-16T11:44:38.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0419999999999996e-06 2024-09-16T11:44:38.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221800+0000", 2024-09-16T11:44:38.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1926999999999999e-05 2024-09-16T11:44:38.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221810+0000", 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0079e-05 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.229980+0000", 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081698469999999992 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230010+0000", 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0746e-05 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.126 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.127 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.127 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33591 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'420, mlcod=53'415)", 2024-09-16T11:44:38.127 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:38.127 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.807002279000002, 2024-09-16T11:44:38.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007702576, 2024-09-16T11:44:38.127 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.127 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.127 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222326+0000", 2024-09-16T11:44:38.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:44:38.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222337+0000", 2024-09-16T11:44:38.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1008e-05 2024-09-16T11:44:38.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222342+0000", 2024-09-16T11:44:38.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0050000000000004e-06 2024-09-16T11:44:38.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222366+0000", 2024-09-16T11:44:38.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3696000000000001e-05 2024-09-16T11:44:38.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222385+0000", 2024-09-16T11:44:38.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-16T11:44:38.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230012+0000", 2024-09-16T11:44:38.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076270940000000001 2024-09-16T11:44:38.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230025+0000", 2024-09-16T11:44:38.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2112e-05 2024-09-16T11:44:38.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.131 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.131 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:55282 4.1e e73/28 4:785299e2:::rbd_data.11c11a49c2da.0000000000000001:head v 73'324, mlcod=73'322)", 2024-09-16T11:44:38.131 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:44:38.131 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.790305760000001, 2024-09-16T11:44:38.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071431890000000003, 2024-09-16T11:44:38.132 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.132 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.132 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:44:38.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:44:38.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239020+0000", 2024-09-16T11:44:38.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.049e-06 2024-09-16T11:44:38.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244238+0000", 2024-09-16T11:44:38.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052183489999999997 2024-09-16T11:44:38.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244252+0000", 2024-09-16T11:44:38.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4246999999999999e-05 2024-09-16T11:44:38.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244275+0000", 2024-09-16T11:44:38.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3093999999999999e-05 2024-09-16T11:44:38.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244314+0000", 2024-09-16T11:44:38.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8495000000000003e-05 2024-09-16T11:44:38.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244330+0000", 2024-09-16T11:44:38.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5917e-05 2024-09-16T11:44:38.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246121+0000", 2024-09-16T11:44:38.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017909239999999999 2024-09-16T11:44:38.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246162+0000", 2024-09-16T11:44:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1114e-05 2024-09-16T11:44:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:61651 4.11 e83/28 4:8f4a51c5:::rbd_data.11c175f8cab7.0000000000000014:head v 83'694, mlcod=81'692)", 2024-09-16T11:44:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:44:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.4158862129999998, 2024-09-16T11:44:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.009466482, 2024-09-16T11:44:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:38.136 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:38.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:38.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:44:38.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:38.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:44:38.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:38.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:38.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613439+0000", 2024-09-16T11:44:38.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8400000000000003e-07 2024-09-16T11:44:38.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:38.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618867+0000", 2024-09-16T11:44:38.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054276059999999998 2024-09-16T11:44:38.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:38.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618883+0000", 2024-09-16T11:44:38.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6265000000000001e-05 2024-09-16T11:44:38.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618890+0000", 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2980000000000003e-06 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618950+0000", 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0025999999999998e-05 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618971+0000", 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0458999999999998e-05 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:38.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622869+0000", 2024-09-16T11:44:38.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038983450000000001 2024-09-16T11:44:38.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:38.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:38.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:38.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622905+0000", 2024-09-16T11:44:38.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599000000000003e-05 2024-09-16T11:44:38.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.140 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:38.140 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:38.141 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:38.142 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:38.143 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:38.140+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:38.240 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:38.240 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:38.240 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:38.240 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:38.242 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:38.241+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:38.248 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:38.343 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:38.341+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:38.345 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:38.345 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:38.345 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:38.345 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:38.345 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:38.353 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:38.445 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:38.443+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:38.492 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:38.492 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:38.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:38.492 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:38.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.492 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89607 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1327104~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.493 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:44:38.493 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.5623256720000001, 2024-09-16T11:44:38.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.033363680999999999, 2024-09-16T11:44:38.493 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.493 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.493 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.494 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.494 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.494 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89607 2024-09-16T11:44:38.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.494 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:44:38.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:44:38.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929100+0000", 2024-09-16T11:44:38.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001870776 2024-09-16T11:44:38.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929104+0000", 2024-09-16T11:44:38.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1160000000000001e-06 2024-09-16T11:44:38.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929118+0000", 2024-09-16T11:44:38.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3891e-05 2024-09-16T11:44:38.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929126+0000", 2024-09-16T11:44:38.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6590000000000007e-06 2024-09-16T11:44:38.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960508+0000", 2024-09-16T11:44:38.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031381920000000001 2024-09-16T11:44:38.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960548+0000", 2024-09-16T11:44:38.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0734e-05 2024-09-16T11:44:38.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960593+0000", 2024-09-16T11:44:38.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4585000000000001e-05 2024-09-16T11:44:38.498 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.498 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.498 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.498 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89611 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1687552~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.498 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:44:38.498 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.5603437119999999, 2024-09-16T11:44:38.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031799929999999997, 2024-09-16T11:44:38.498 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.498 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.498 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89611 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929212+0000", 2024-09-16T11:44:38.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0800000000000001e-07 2024-09-16T11:44:38.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929213+0000", 2024-09-16T11:44:38.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6509999999999999e-06 2024-09-16T11:44:38.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929218+0000", 2024-09-16T11:44:38.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9899999999999997e-06 2024-09-16T11:44:38.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929221+0000", 2024-09-16T11:44:38.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1140000000000002e-06 2024-09-16T11:44:38.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960926+0000", 2024-09-16T11:44:38.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031704436000000003 2024-09-16T11:44:38.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960966+0000", 2024-09-16T11:44:38.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0055999999999997e-05 2024-09-16T11:44:38.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961011+0000", 2024-09-16T11:44:38.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5374999999999998e-05 2024-09-16T11:44:38.506 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.506 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.506 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.507 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89612 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1777664~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.507 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:38.507 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.5603305029999999, 2024-09-16T11:44:38.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031893013999999997, 2024-09-16T11:44:38.508 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.508 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.508 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.508 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.508 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.508 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89612 2024-09-16T11:44:38.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.509 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:38.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:38.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:38.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6e-07 2024-09-16T11:44:38.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929226+0000", 2024-09-16T11:44:38.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5099999999999999e-06 2024-09-16T11:44:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929231+0000", 2024-09-16T11:44:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4889999999999997e-06 2024-09-16T11:44:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929234+0000", 2024-09-16T11:44:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7410000000000001e-06 2024-09-16T11:44:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961033+0000", 2024-09-16T11:44:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031799279 2024-09-16T11:44:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961071+0000", 2024-09-16T11:44:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7759000000000002e-05 2024-09-16T11:44:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961118+0000", 2024-09-16T11:44:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6975999999999999e-05 2024-09-16T11:44:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89613 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1867776~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.560318364, 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031996359000000002, 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.515 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89613 2024-09-16T11:44:38.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.516 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:38.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:38.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:38.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8999999999999998e-07 2024-09-16T11:44:38.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929243+0000", 2024-09-16T11:44:38.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8719999999999999e-06 2024-09-16T11:44:38.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929249+0000", 2024-09-16T11:44:38.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6229999999999999e-06 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929253+0000", 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961145+0000", 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031892648000000003 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961188+0000", 2024-09-16T11:44:38.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2466999999999998e-05 2024-09-16T11:44:38.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961233+0000", 2024-09-16T11:44:38.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5049999999999997e-05 2024-09-16T11:44:38.519 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.519 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.519 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.520 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89615 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1957888~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.520 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:38.520 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.560298494, 2024-09-16T11:44:38.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032076532999999997, 2024-09-16T11:44:38.520 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.520 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.520 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.520 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.520 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.520 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89615 2024-09-16T11:44:38.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929259+0000", 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7269999999999999e-06 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929264+0000", 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.524e-06 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929267+0000", 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.994e-06 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961252+0000", 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031985188999999997 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961291+0000", 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8263000000000003e-05 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.523 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961333+0000", 2024-09-16T11:44:38.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2491999999999999e-05 2024-09-16T11:44:38.524 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.524 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.524 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.524 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89617 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 2048000~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.524 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:38.524 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.560284861, 2024-09-16T11:44:38.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032185244000000002, 2024-09-16T11:44:38.524 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.524 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.524 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.525 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.525 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.525 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89617 2024-09-16T11:44:38.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.525 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.525 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:38.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.525 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0499999999999999e-07 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929272+0000", 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.615e-06 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929279+0000", 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5459999999999997e-06 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929282+0000", 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.472e-06 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961361+0000", 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032078441999999999 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961401+0000", 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0425e-05 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961455+0000", 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4438999999999998e-05 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.528 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89637 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4235264~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.529 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:44:38.529 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.5602646039999999, 2024-09-16T11:44:38.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032120463000000002, 2024-09-16T11:44:38.529 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.529 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.529 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89637 2024-09-16T11:44:38.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.529 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929291+0000", 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:44:38.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929292+0000", 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3120000000000001e-06 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929295+0000", 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3349999999999998e-06 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929297+0000", 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0549999999999998e-06 2024-09-16T11:44:38.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.532 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961346+0000", 2024-09-16T11:44:38.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032048920000000002 2024-09-16T11:44:38.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.532 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961379+0000", 2024-09-16T11:44:38.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2533000000000002e-05 2024-09-16T11:44:38.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.532 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961411+0000", 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1921000000000002e-05 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89639 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4325376~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.5602555250000001, 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032194749000000002, 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.533 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.534 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.534 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89639 2024-09-16T11:44:38.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.534 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:44:38.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:44:38.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929300+0000", 2024-09-16T11:44:38.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:44:38.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929301+0000", 2024-09-16T11:44:38.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.018e-06 2024-09-16T11:44:38.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929304+0000", 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8890000000000002e-06 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929306+0000", 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.883e-06 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961422+0000", 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032116899999999997 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.536 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961460+0000", 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7265999999999999e-05 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961494+0000", 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4499000000000003e-05 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89641 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4415488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.537 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.560247602, 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032287719999999999, 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89641 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929308+0000", 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1500000000000001e-07 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929309+0000", 2024-09-16T11:44:38.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.02e-06 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929312+0000", 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3000000000000002e-06 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929314+0000", 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0449999999999999e-06 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961521+0000", 2024-09-16T11:44:38.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032206835000000003 2024-09-16T11:44:38.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961561+0000", 2024-09-16T11:44:38.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0580000000000001e-05 2024-09-16T11:44:38.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961595+0000", 2024-09-16T11:44:38.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3725e-05 2024-09-16T11:44:38.541 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.541 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89643 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4505600~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.560239164, 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032377892999999998, 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89643 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0100000000000001e-07 2024-09-16T11:44:38.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929317+0000", 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.161e-06 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929320+0000", 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1580000000000001e-06 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929322+0000", 2024-09-16T11:44:38.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9929999999999998e-06 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961619+0000", 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032296392 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961659+0000", 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9805000000000003e-05 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961694+0000", 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5182999999999998e-05 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89644 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4243456~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.5602307849999999, 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032467912000000002, 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.546 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89644 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0100000000000001e-07 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929325+0000", 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9899999999999999e-07 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929328+0000", 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7539999999999998e-06 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929330+0000", 2024-09-16T11:44:38.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3690000000000001e-06 2024-09-16T11:44:38.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961725+0000", 2024-09-16T11:44:38.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032394088000000001 2024-09-16T11:44:38.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961759+0000", 2024-09-16T11:44:38.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4316999999999997e-05 2024-09-16T11:44:38.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961792+0000", 2024-09-16T11:44:38.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3284000000000002e-05 2024-09-16T11:44:38.550 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.550 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.550 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.550 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89645 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4595712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.5602227129999999, 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032554313000000001, 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89645 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:38.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:38.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:38.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.99e-07 2024-09-16T11:44:38.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929333+0000", 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9299999999999996e-07 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929336+0000", 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8459999999999999e-06 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929340+0000", 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6370000000000001e-06 2024-09-16T11:44:38.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961814+0000", 2024-09-16T11:44:38.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032474461000000003 2024-09-16T11:44:38.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961852+0000", 2024-09-16T11:44:38.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7877999999999999e-05 2024-09-16T11:44:38.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961887+0000", 2024-09-16T11:44:38.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4399e-05 2024-09-16T11:44:38.555 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.555 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.555 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.555 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89646 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4333568~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.555 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:38.555 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.5602130760000001, 2024-09-16T11:44:38.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032641146000000003, 2024-09-16T11:44:38.556 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.556 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.556 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89646 2024-09-16T11:44:38.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.556 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.556 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:38.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:38.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:38.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.04e-07 2024-09-16T11:44:38.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929343+0000", 2024-09-16T11:44:38.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7000000000000003e-07 2024-09-16T11:44:38.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929346+0000", 2024-09-16T11:44:38.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7350000000000001e-06 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929349+0000", 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7030000000000002e-06 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961916+0000", 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032567634999999998 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961950+0000", 2024-09-16T11:44:38.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3510000000000003e-05 2024-09-16T11:44:38.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.560 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961983+0000", 2024-09-16T11:44:38.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3389000000000002e-05 2024-09-16T11:44:38.560 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.560 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.560 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89647 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4685824~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:44:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.560204674, 2024-09-16T11:44:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032727342, 2024-09-16T11:44:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89647 2024-09-16T11:44:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.561 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:44:38.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:44:38.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.563 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931111+0000", 2024-09-16T11:44:38.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001760462 2024-09-16T11:44:38.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931114+0000", 2024-09-16T11:44:38.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.67e-06 2024-09-16T11:44:38.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931122+0000", 2024-09-16T11:44:38.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6059999999999997e-06 2024-09-16T11:44:38.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931126+0000", 2024-09-16T11:44:38.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-06 2024-09-16T11:44:38.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962005+0000", 2024-09-16T11:44:38.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.030879125 2024-09-16T11:44:38.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.567 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962044+0000", 2024-09-16T11:44:38.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8779000000000003e-05 2024-09-16T11:44:38.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962078+0000", 2024-09-16T11:44:38.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4010000000000001e-05 2024-09-16T11:44:38.568 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.568 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.568 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.569 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89656 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4784128~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.569 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:44:38.569 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.558354625, 2024-09-16T11:44:38.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031742976999999999, 2024-09-16T11:44:38.570 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.570 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.570 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.570 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.570 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.571 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89656 2024-09-16T11:44:38.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.571 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.572 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:44:38.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.572 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.572 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:44:38.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931201+0000", 2024-09-16T11:44:38.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0699999999999999e-07 2024-09-16T11:44:38.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931202+0000", 2024-09-16T11:44:38.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5900000000000005e-07 2024-09-16T11:44:38.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931204+0000", 2024-09-16T11:44:38.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8439999999999998e-06 2024-09-16T11:44:38.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931206+0000", 2024-09-16T11:44:38.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.871e-06 2024-09-16T11:44:38.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962877+0000", 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031670831000000003 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962910+0000", 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3405999999999998e-05 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962943+0000", 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2858999999999998e-05 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.576 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.577 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.577 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89657 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5136384~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.577 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:38.577 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.5583468140000001, 2024-09-16T11:44:38.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031827938, 2024-09-16T11:44:38.577 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.577 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.577 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.577 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.577 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.577 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89657 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.28e-07 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931209+0000", 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9099999999999991e-07 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931213+0000", 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7639999999999999e-06 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931217+0000", 2024-09-16T11:44:38.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4759999999999998e-06 2024-09-16T11:44:38.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962964+0000", 2024-09-16T11:44:38.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031747598000000002 2024-09-16T11:44:38.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963003+0000", 2024-09-16T11:44:38.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8287000000000001e-05 2024-09-16T11:44:38.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.581 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.581 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963036+0000", 2024-09-16T11:44:38.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3593999999999997e-05 2024-09-16T11:44:38.581 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.581 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.581 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.581 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89658 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4874240~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.581 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:38.582 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.558319319, 2024-09-16T11:44:38.582 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031905067000000002, 2024-09-16T11:44:38.582 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.582 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.582 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.582 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.582 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.582 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89658 2024-09-16T11:44:38.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.582 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4700000000000001e-07 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931238+0000", 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931245+0000", 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.64e-06 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931248+0000", 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5719999999999999e-06 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963066+0000", 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031817326999999999 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963107+0000", 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1220000000000002e-05 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963141+0000", 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3896999999999997e-05 2024-09-16T11:44:38.585 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89659 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4964352~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.5583030170000001, 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031984460999999999, 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89659 2024-09-16T11:44:38.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.587 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931254+0000", 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.776e-06 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931262+0000", 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1680000000000004e-06 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931266+0000", 2024-09-16T11:44:38.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.732e-06 2024-09-16T11:44:38.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963165+0000", 2024-09-16T11:44:38.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031898693999999998 2024-09-16T11:44:38.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963202+0000", 2024-09-16T11:44:38.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7517e-05 2024-09-16T11:44:38.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963236+0000", 2024-09-16T11:44:38.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4230000000000003e-05 2024-09-16T11:44:38.590 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.590 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.590 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.591 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89660 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5226496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.591 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:38.592 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.5582849969999999, 2024-09-16T11:44:38.592 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032062368000000001, 2024-09-16T11:44:38.592 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.592 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.592 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.592 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.593 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.593 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89660 2024-09-16T11:44:38.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.593 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.593 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.594 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:38.594 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.594 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.594 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.594 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:38.594 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:38.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.39e-07 2024-09-16T11:44:38.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931272+0000", 2024-09-16T11:44:38.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:44:38.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931279+0000", 2024-09-16T11:44:38.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:44:38.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931282+0000", 2024-09-16T11:44:38.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.721e-06 2024-09-16T11:44:38.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963266+0000", 2024-09-16T11:44:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031983971999999999 2024-09-16T11:44:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963300+0000", 2024-09-16T11:44:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3318000000000002e-05 2024-09-16T11:44:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963332+0000", 2024-09-16T11:44:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2768000000000002e-05 2024-09-16T11:44:38.597 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89661 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5054464~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 1.5582685729999999, 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032139394000000002, 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89661 2024-09-16T11:44:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:44:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:44:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933104+0000", 2024-09-16T11:44:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00181734 2024-09-16T11:44:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933107+0000", 2024-09-16T11:44:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5140000000000002e-06 2024-09-16T11:44:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933115+0000", 2024-09-16T11:44:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:44:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933124+0000", 2024-09-16T11:44:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8910000000000001e-06 2024-09-16T11:44:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963354+0000", 2024-09-16T11:44:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.030230825999999999 2024-09-16T11:44:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963392+0000", 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7765000000000002e-05 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963426+0000", 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3603000000000003e-05 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:38.603 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:38.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:38.604 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:38.602+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:38.705 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:38.703+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:38.742 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:38.742 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:38.742 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:38.742 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:38.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:38.805 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:38.804+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:38.890 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:38.890 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:38.890 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:38.890 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:38.890 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:38.898 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:38.906 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:38.905+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:38.996 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:38.997 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:38.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:38.997 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:38.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.997 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19364 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1171456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:38.997 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:38.997 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.480578946000001, 2024-09-16T11:44:38.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049001805000000002, 2024-09-16T11:44:38.997 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:38.997 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:38.997 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:38.997 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19364 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:38.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513280+0000", 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7090000000000001e-06 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513286+0000", 2024-09-16T11:44:38.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5840000000000002e-06 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513290+0000", 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3680000000000004e-06 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562121+0000", 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048831324000000002 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562195+0000", 2024-09-16T11:44:39.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3415000000000001e-05 2024-09-16T11:44:39.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562280+0000", 2024-09-16T11:44:39.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4975999999999994e-05 2024-09-16T11:44:39.001 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.001 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.001 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.002 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19365 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1490944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.002 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:39.002 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.480559206999999, 2024-09-16T11:44:39.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049335349000000001, 2024-09-16T11:44:39.002 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.002 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.002 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.002 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.002 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.002 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19365 2024-09-16T11:44:39.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.002 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:39.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513300+0000", 2024-09-16T11:44:39.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3020000000000002e-06 2024-09-16T11:44:39.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513306+0000", 2024-09-16T11:44:39.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8780000000000003e-06 2024-09-16T11:44:39.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513310+0000", 2024-09-16T11:44:39.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1309999999999999e-06 2024-09-16T11:44:39.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562322+0000", 2024-09-16T11:44:39.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049011635999999997 2024-09-16T11:44:39.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562544+0000", 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022168000000000001 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562633+0000", 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9350000000000003e-05 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19366 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1916928~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:39.006 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.480542274999998, 2024-09-16T11:44:39.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049865073000000003, 2024-09-16T11:44:39.007 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.007 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.007 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.007 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.007 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.007 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19366 2024-09-16T11:44:39.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.007 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:39.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515102+0000", 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017877419999999999 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515106+0000", 2024-09-16T11:44:39.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4450000000000001e-06 2024-09-16T11:44:39.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515113+0000", 2024-09-16T11:44:39.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7680000000000008e-06 2024-09-16T11:44:39.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515117+0000", 2024-09-16T11:44:39.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:44:39.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562756+0000", 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047638579 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563093+0000", 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000337516 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563180+0000", 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6420000000000003e-05 2024-09-16T11:44:39.010 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.011 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.011 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.011 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19371 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3194880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.011 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:39.011 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.478645067999999, 2024-09-16T11:44:39.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048948356999999998, 2024-09-16T11:44:39.011 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.011 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.011 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.011 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.011 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19371 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:39.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:39.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515214+0000", 2024-09-16T11:44:39.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5969999999999999e-06 2024-09-16T11:44:39.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515219+0000", 2024-09-16T11:44:39.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4559999999999999e-06 2024-09-16T11:44:39.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515223+0000", 2024-09-16T11:44:39.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:44:39.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563991+0000", 2024-09-16T11:44:39.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048768911999999998 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564063+0000", 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2010999999999999e-05 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564160+0000", 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6617e-05 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.016 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19372 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3620864~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.016 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:39.016 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.478630850999998, 2024-09-16T11:44:39.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049130502, 2024-09-16T11:44:39.016 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.016 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.016 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.016 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.016 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.016 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19372 2024-09-16T11:44:39.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.016 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.017 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.017 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:39.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.017 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.017 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:39.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.017 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515228+0000", 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515233+0000", 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515237+0000", 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.27e-06 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564199+0000", 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048961787999999999 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564275+0000", 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6001000000000006e-05 2024-09-16T11:44:39.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564356+0000", 2024-09-16T11:44:39.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1941999999999996e-05 2024-09-16T11:44:39.020 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.020 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.020 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.020 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19373 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3940352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.020 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:39.020 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.478616451000001, 2024-09-16T11:44:39.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049289588000000002, 2024-09-16T11:44:39.021 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.021 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.021 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.021 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.021 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.021 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19373 2024-09-16T11:44:39.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.021 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.021 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.021 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:39.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:39.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515241+0000", 2024-09-16T11:44:39.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:44:39.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515242+0000", 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.528e-06 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515248+0000", 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515251+0000", 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2480000000000001e-06 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564388+0000", 2024-09-16T11:44:39.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049137313000000002 2024-09-16T11:44:39.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564455+0000", 2024-09-16T11:44:39.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7049000000000001e-05 2024-09-16T11:44:39.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564530+0000", 2024-09-16T11:44:39.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4626e-05 2024-09-16T11:44:39.025 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.025 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.025 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.025 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19374 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 4046848~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.025 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:39.025 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.478602047999999, 2024-09-16T11:44:39.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049457133, 2024-09-16T11:44:39.025 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.025 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.026 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.026 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.026 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.026 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19374 2024-09-16T11:44:39.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.026 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:39.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515257+0000", 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5740000000000001e-06 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515262+0000", 2024-09-16T11:44:39.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2399999999999998e-06 2024-09-16T11:44:39.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515265+0000", 2024-09-16T11:44:39.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3909999999999998e-06 2024-09-16T11:44:39.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564555+0000", 2024-09-16T11:44:39.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049290002999999999 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564628+0000", 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2559999999999996e-05 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564712+0000", 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4040000000000002e-05 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19375 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3301376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.478587904999998, 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049606631999999998, 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19375 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:39.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:39.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:39.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515271+0000", 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5230000000000001e-06 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515276+0000", 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4650000000000002e-06 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515279+0000", 2024-09-16T11:44:39.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0630000000000002e-06 2024-09-16T11:44:39.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.033 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564738+0000", 2024-09-16T11:44:39.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049459001000000002 2024-09-16T11:44:39.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.033 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564780+0000", 2024-09-16T11:44:39.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1693999999999998e-05 2024-09-16T11:44:39.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564876+0000", 2024-09-16T11:44:39.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5571999999999999e-05 2024-09-16T11:44:39.034 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.034 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.034 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.034 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19376 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3727360~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.035 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:39.035 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.478573992999998, 2024-09-16T11:44:39.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049720590000000002, 2024-09-16T11:44:39.035 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.035 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.035 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.036 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.036 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.036 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19376 2024-09-16T11:44:39.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.036 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:39.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:39.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517103+0000", 2024-09-16T11:44:39.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001820276 2024-09-16T11:44:39.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517108+0000", 2024-09-16T11:44:39.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5410000000000002e-06 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517120+0000", 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2092e-05 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517125+0000", 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6829999999999996e-06 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564870+0000", 2024-09-16T11:44:39.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047745040000000002 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564921+0000", 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0538999999999998e-05 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565003+0000", 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2418999999999998e-05 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19492 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 434176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.476645136999998, 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048926948999999997, 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19492 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:39.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:39.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:39.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3099999999999999e-07 2024-09-16T11:44:39.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517213+0000", 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6599999999999994e-07 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517216+0000", 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.706e-06 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517218+0000", 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.035e-06 2024-09-16T11:44:39.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565959+0000", 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048741380000000001 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566036+0000", 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7362999999999998e-05 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566139+0000", 2024-09-16T11:44:39.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102268 2024-09-16T11:44:39.044 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.044 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.044 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.044 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19493 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 860160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.044 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:39.044 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.476623465999999, 2024-09-16T11:44:39.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049086604999999998, 2024-09-16T11:44:39.044 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.044 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.044 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.044 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.045 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.045 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19493 2024-09-16T11:44:39.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.045 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.045 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.045 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:39.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517234+0000", 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.37e-07 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517236+0000", 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7150000000000001e-06 2024-09-16T11:44:39.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517243+0000", 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9489999999999999e-06 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517246+0000", 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.636e-06 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566163+0000", 2024-09-16T11:44:39.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048917284999999998 2024-09-16T11:44:39.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566237+0000", 2024-09-16T11:44:39.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3196999999999997e-05 2024-09-16T11:44:39.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566320+0000", 2024-09-16T11:44:39.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3286000000000002e-05 2024-09-16T11:44:39.048 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.048 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.048 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19494 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1179648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.476602849999999, 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049259562999999999, 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19494 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.049 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:39.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517256+0000", 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.888e-06 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517265+0000", 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8629999999999997e-06 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517269+0000", 2024-09-16T11:44:39.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2039999999999999e-06 2024-09-16T11:44:39.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566352+0000", 2024-09-16T11:44:39.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049083107000000001 2024-09-16T11:44:39.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566429+0000", 2024-09-16T11:44:39.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7025000000000002e-05 2024-09-16T11:44:39.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566514+0000", 2024-09-16T11:44:39.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4121000000000003e-05 2024-09-16T11:44:39.053 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.053 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.053 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.053 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19495 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1286144~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.053 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:39.053 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.476583427000001, 2024-09-16T11:44:39.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049492880000000003, 2024-09-16T11:44:39.053 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19495 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:39.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.055 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517274+0000", 2024-09-16T11:44:39.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:44:39.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.055 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517276+0000", 2024-09-16T11:44:39.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8950000000000001e-06 2024-09-16T11:44:39.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517281+0000", 2024-09-16T11:44:39.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3879999999999999e-06 2024-09-16T11:44:39.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517285+0000", 2024-09-16T11:44:39.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5930000000000002e-06 2024-09-16T11:44:39.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566579+0000", 2024-09-16T11:44:39.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049294175000000003 2024-09-16T11:44:39.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566670+0000", 2024-09-16T11:44:39.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1096999999999999e-05 2024-09-16T11:44:39.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566766+0000", 2024-09-16T11:44:39.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6355999999999997e-05 2024-09-16T11:44:39.058 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.058 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.058 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.058 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19496 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1605632~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.058 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:39.058 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.476568342, 2024-09-16T11:44:39.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049632981999999999, 2024-09-16T11:44:39.059 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.059 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.059 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.059 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.059 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.059 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19496 2024-09-16T11:44:39.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.060 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:39.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:39.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517289+0000", 2024-09-16T11:44:39.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:39.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517290+0000", 2024-09-16T11:44:39.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.581e-06 2024-09-16T11:44:39.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517300+0000", 2024-09-16T11:44:39.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:39.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517304+0000", 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566789+0000", 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049485102000000003 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566843+0000", 2024-09-16T11:44:39.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3937000000000002e-05 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566921+0000", 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8397000000000002e-05 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19497 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 114688~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.476548647000001, 2024-09-16T11:44:39.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049761367000000001, 2024-09-16T11:44:39.064 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.064 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.064 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.064 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.064 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.064 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19497 2024-09-16T11:44:39.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.064 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.064 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:39.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519116+0000", 2024-09-16T11:44:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018079439999999999 2024-09-16T11:44:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519120+0000", 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2760000000000002e-06 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519134+0000", 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3468999999999999e-05 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519140+0000", 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.516e-06 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.067 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566943+0000", 2024-09-16T11:44:39.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047802561 2024-09-16T11:44:39.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.067 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.067 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566982+0000", 2024-09-16T11:44:39.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9269e-05 2024-09-16T11:44:39.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.067 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.067 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567070+0000", 2024-09-16T11:44:39.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7331999999999997e-05 2024-09-16T11:44:39.067 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19504 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 966656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.474620160000001, 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049045863000000002, 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.068 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19504 2024-09-16T11:44:39.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.069 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:39.069 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:39.069 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519239+0000", 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.962e-06 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519244+0000", 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5480000000000001e-06 2024-09-16T11:44:39.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519248+0000", 2024-09-16T11:44:39.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.641e-06 2024-09-16T11:44:39.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568150+0000", 2024-09-16T11:44:39.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048901864000000003 2024-09-16T11:44:39.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568208+0000", 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8399000000000001e-05 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568283+0000", 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4105999999999998e-05 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19505 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1392640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:39.072 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.474604855000003, 2024-09-16T11:44:39.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049202304000000002, 2024-09-16T11:44:39.073 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.073 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.073 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.073 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.073 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.073 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19505 2024-09-16T11:44:39.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.073 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:39.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519254+0000", 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.761e-06 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519257+0000", 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.343e-06 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519259+0000", 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0559999999999999e-06 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568307+0000", 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049047445000000002 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568382+0000", 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5574999999999994e-05 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568454+0000", 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1799999999999997e-05 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.076 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.077 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.077 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19506 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1712128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.077 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:39.077 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.474594817000003, 2024-09-16T11:44:39.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049341532, 2024-09-16T11:44:39.077 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.077 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.077 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.077 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.077 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.077 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19506 2024-09-16T11:44:39.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.078 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:39.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:39.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521102+0000", 2024-09-16T11:44:39.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018404700000000001 2024-09-16T11:44:39.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521106+0000", 2024-09-16T11:44:39.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8820000000000003e-06 2024-09-16T11:44:39.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521119+0000", 2024-09-16T11:44:39.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2546e-05 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521125+0000", 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0249999999999999e-06 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568484+0000", 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047359120999999997 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.080 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568542+0000", 2024-09-16T11:44:39.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8360999999999999e-05 2024-09-16T11:44:39.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568604+0000", 2024-09-16T11:44:39.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1126999999999999e-05 2024-09-16T11:44:39.081 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.081 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.081 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.081 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19515 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1499136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:39.081 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:39.082 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 43.472602532000003, 2024-09-16T11:44:39.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048949431000000002, 2024-09-16T11:44:39.082 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.082 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:39.082 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.082 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.082 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.082 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19515 2024-09-16T11:44:39.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.082 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523130+0000", 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00187604 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.083 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523134+0000", 2024-09-16T11:44:39.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:44:39.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.084 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523144+0000", 2024-09-16T11:44:39.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2010000000000005e-06 2024-09-16T11:44:39.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.084 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523149+0000", 2024-09-16T11:44:39.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0039999999999999e-06 2024-09-16T11:44:39.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570113+0000", 2024-09-16T11:44:39.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.046964517999999997 2024-09-16T11:44:39.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570157+0000", 2024-09-16T11:44:39.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4056e-05 2024-09-16T11:44:39.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570204+0000", 2024-09-16T11:44:39.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6591999999999997e-05 2024-09-16T11:44:39.086 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.086 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.086 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.086 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:44111 4.8 4:17f3f47e:::rbd_data.11c1e170f5f0.000000000000002f:head [delete] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:44:39.086 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:39.086 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 29.010622251000001, 2024-09-16T11:44:39.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93903426400000001, 2024-09-16T11:44:39.086 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:39.086 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 44111 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:39.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7199999999999998e-07 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983239+0000", 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9879999999999998e-06 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:39.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983248+0000", 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.371e-06 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983253+0000", 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1150000000000002e-06 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983303+0000", 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0380000000000002e-05 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983311+0000", 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7519999999999996e-06 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920618+0000", 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93730672000000004 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920730+0000", 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011265900000000001 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920842+0000", 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111638 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.921503+0000", 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066074299999999999 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922208+0000", 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00070492899999999997 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922233+0000", 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5922999999999999e-05 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922269+0000", 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5373999999999997e-05 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:39.092 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:39.093 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:44:39.094 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:39.092+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:39.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:39.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:44:39.195 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:39.193+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:39.237 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:39.237 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:39.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:39.296 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:39.294+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:39.301 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:39.331 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:39.331 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:39.349 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:39.349 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:39.349 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:39.349 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:39.349 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:39.358 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:39.362 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:39.362 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:39.393 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:39.393 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:39.396 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:39.394+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:39.423 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:39.423 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:39.455 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:39.456 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:39.468 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:39.468 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:39.477 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:39.487 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:39.488 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:39.503 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:39.503 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:39.504 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:39.501+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:39.586 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:39.586 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:39.594 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:39.639 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:39.637+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:39.692 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:39.692 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:39.698 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:39.697+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:39.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:39.799 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:39.797+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:39.840 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:39.840 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:39.846 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:39.900 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:39.899+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:39.957 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:39.957 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:39.966 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:40.006 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:40.005+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:40.044 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T11:44:40.102 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:40.100+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:40.112 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:40.112 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:40.122 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:40.203 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:40.201+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:40.220 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:40.220 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:40.229 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:40.304 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:40.302+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:40.327 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:40.327 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:40.336 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:40.405 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:40.404+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:40.435 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:40.435 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:40.443 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:40.528 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:40.527+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:40.582 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:40.582 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:40.591 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:40.632 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:40.631+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:40.707 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:40.705+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:40.730 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:40.730 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:40.738 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:40.809 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:40.807+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:40.835 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:40.835 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:40.843 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:40.910 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:40.908+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:40.982 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:40.982 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:40.991 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:41.034 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:41.033+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:41.101 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:41.101 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:41.110 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:41.151 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:41.150+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:41.213 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:41.212+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:41.248 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:41.248 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:41.256 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:41.340 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:41.339+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:41.394 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:41.394 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:41.402 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:41.443 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:41.442+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:41.505 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:41.505 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:41.513 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:41.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:44:41.516 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:41.515+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:41.614 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:41.614 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:41.616 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:41.614+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:41.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:41.629 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:41.629 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:41.646 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:41.646 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:41.661 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:41.661 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:41.676 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:41.677 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:41.693 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:41.693 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:41.709 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:41.709 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:41.717 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:41.716+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:41.724 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:41.724 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:41.730 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:41.731 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:41.731 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:41.731 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:41.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.731 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89607 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1327104~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.731 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:44:41.731 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.8008381379999996, 2024-09-16T11:44:41.731 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.033363680999999999, 2024-09-16T11:44:41.731 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.731 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.731 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.731 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.731 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.732 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89607 2024-09-16T11:44:41.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.732 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.732 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.732 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:44:41.732 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.732 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.732 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:44:41.732 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.733 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.733 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929100+0000", 2024-09-16T11:44:41.733 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001870776 2024-09-16T11:44:41.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.733 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.733 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929104+0000", 2024-09-16T11:44:41.733 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1160000000000001e-06 2024-09-16T11:44:41.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.733 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.733 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929118+0000", 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3891e-05 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929126+0000", 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6590000000000007e-06 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960508+0000", 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031381920000000001 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.734 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960548+0000", 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0734e-05 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960593+0000", 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4585000000000001e-05 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89611 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1687552~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.735 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:44:41.736 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7988561780000003, 2024-09-16T11:44:41.736 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031799929999999997, 2024-09-16T11:44:41.736 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.736 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.736 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.736 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.736 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.736 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89611 2024-09-16T11:44:41.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.736 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.736 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.736 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.737 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:44:41.737 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.737 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.737 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:44:41.737 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.737 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.737 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929212+0000", 2024-09-16T11:44:41.737 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0800000000000001e-07 2024-09-16T11:44:41.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.738 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.738 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929213+0000", 2024-09-16T11:44:41.738 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6509999999999999e-06 2024-09-16T11:44:41.738 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.738 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.738 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929218+0000", 2024-09-16T11:44:41.738 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9899999999999997e-06 2024-09-16T11:44:41.738 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.738 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.739 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929221+0000", 2024-09-16T11:44:41.739 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1140000000000002e-06 2024-09-16T11:44:41.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.739 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.739 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960926+0000", 2024-09-16T11:44:41.739 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031704436000000003 2024-09-16T11:44:41.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.739 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.739 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960966+0000", 2024-09-16T11:44:41.739 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0055999999999997e-05 2024-09-16T11:44:41.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.740 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.740 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.740 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961011+0000", 2024-09-16T11:44:41.740 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5374999999999998e-05 2024-09-16T11:44:41.740 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.740 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.740 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.740 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.740 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89612 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1777664~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.740 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:41.740 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7988429689999998, 2024-09-16T11:44:41.741 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031893013999999997, 2024-09-16T11:44:41.741 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.741 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.741 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.741 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.741 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.741 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89612 2024-09-16T11:44:41.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.741 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.741 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.741 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:41.741 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.742 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.742 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:41.742 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.742 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.742 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:41.742 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6e-07 2024-09-16T11:44:41.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.742 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.743 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929226+0000", 2024-09-16T11:44:41.743 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5099999999999999e-06 2024-09-16T11:44:41.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.743 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.743 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929231+0000", 2024-09-16T11:44:41.743 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4889999999999997e-06 2024-09-16T11:44:41.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.743 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.743 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929234+0000", 2024-09-16T11:44:41.743 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7410000000000001e-06 2024-09-16T11:44:41.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.744 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.744 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961033+0000", 2024-09-16T11:44:41.744 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031799279 2024-09-16T11:44:41.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.744 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.744 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961071+0000", 2024-09-16T11:44:41.744 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7759000000000002e-05 2024-09-16T11:44:41.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.744 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.745 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961118+0000", 2024-09-16T11:44:41.745 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6975999999999999e-05 2024-09-16T11:44:41.745 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.745 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.745 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.745 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.745 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.745 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89613 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1867776~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.745 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:41.745 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.79883083, 2024-09-16T11:44:41.745 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031996359000000002, 2024-09-16T11:44:41.745 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89613 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.746 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.747 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:41.747 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.747 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.747 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:41.747 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8999999999999998e-07 2024-09-16T11:44:41.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.747 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.747 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929243+0000", 2024-09-16T11:44:41.747 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8719999999999999e-06 2024-09-16T11:44:41.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929249+0000", 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6229999999999999e-06 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929253+0000", 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961145+0000", 2024-09-16T11:44:41.748 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031892648000000003 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961188+0000", 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2466999999999998e-05 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961233+0000", 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5049999999999997e-05 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89615 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1957888~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.79881096, 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032076532999999997, 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89615 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.750 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.751 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.751 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:41.751 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.751 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.751 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:41.751 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.751 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.751 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:41.751 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929259+0000", 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7269999999999999e-06 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929264+0000", 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.524e-06 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929267+0000", 2024-09-16T11:44:41.752 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.994e-06 2024-09-16T11:44:41.753 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.753 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.753 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.753 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961252+0000", 2024-09-16T11:44:41.753 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031985188999999997 2024-09-16T11:44:41.753 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.753 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.753 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.753 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961291+0000", 2024-09-16T11:44:41.753 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8263000000000003e-05 2024-09-16T11:44:41.753 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961333+0000", 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2491999999999999e-05 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89617 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 2048000~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7987973269999999, 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032185244000000002, 2024-09-16T11:44:41.754 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.755 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.755 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.755 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.755 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.755 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89617 2024-09-16T11:44:41.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.755 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.755 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.755 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.755 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:41.755 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.755 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0499999999999999e-07 2024-09-16T11:44:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929272+0000", 2024-09-16T11:44:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.615e-06 2024-09-16T11:44:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929279+0000", 2024-09-16T11:44:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5459999999999997e-06 2024-09-16T11:44:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929282+0000", 2024-09-16T11:44:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.472e-06 2024-09-16T11:44:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961361+0000", 2024-09-16T11:44:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032078441999999999 2024-09-16T11:44:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961401+0000", 2024-09-16T11:44:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0425e-05 2024-09-16T11:44:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961455+0000", 2024-09-16T11:44:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4438999999999998e-05 2024-09-16T11:44:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89637 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4235264~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:44:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7987770699999999, 2024-09-16T11:44:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032120463000000002, 2024-09-16T11:44:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89637 2024-09-16T11:44:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:44:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:44:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929291+0000", 2024-09-16T11:44:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:44:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929292+0000", 2024-09-16T11:44:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3120000000000001e-06 2024-09-16T11:44:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929295+0000", 2024-09-16T11:44:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3349999999999998e-06 2024-09-16T11:44:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929297+0000", 2024-09-16T11:44:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0549999999999998e-06 2024-09-16T11:44:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961346+0000", 2024-09-16T11:44:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032048920000000002 2024-09-16T11:44:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961379+0000", 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2533000000000002e-05 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961411+0000", 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1921000000000002e-05 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89639 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4325376~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:44:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7987679910000001, 2024-09-16T11:44:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032194749000000002, 2024-09-16T11:44:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89639 2024-09-16T11:44:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929300+0000", 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929301+0000", 2024-09-16T11:44:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.018e-06 2024-09-16T11:44:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929304+0000", 2024-09-16T11:44:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8890000000000002e-06 2024-09-16T11:44:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929306+0000", 2024-09-16T11:44:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.883e-06 2024-09-16T11:44:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961422+0000", 2024-09-16T11:44:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032116899999999997 2024-09-16T11:44:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961460+0000", 2024-09-16T11:44:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7265999999999999e-05 2024-09-16T11:44:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961494+0000", 2024-09-16T11:44:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4499000000000003e-05 2024-09-16T11:44:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89641 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4415488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:44:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.798760068, 2024-09-16T11:44:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032287719999999999, 2024-09-16T11:44:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89641 2024-09-16T11:44:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:44:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:44:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929308+0000", 2024-09-16T11:44:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1500000000000001e-07 2024-09-16T11:44:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929309+0000", 2024-09-16T11:44:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.02e-06 2024-09-16T11:44:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929312+0000", 2024-09-16T11:44:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3000000000000002e-06 2024-09-16T11:44:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929314+0000", 2024-09-16T11:44:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0449999999999999e-06 2024-09-16T11:44:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961521+0000", 2024-09-16T11:44:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032206835000000003 2024-09-16T11:44:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961561+0000", 2024-09-16T11:44:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0580000000000001e-05 2024-09-16T11:44:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961595+0000", 2024-09-16T11:44:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3725e-05 2024-09-16T11:44:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89643 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4505600~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7987516299999999, 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032377892999999998, 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89643 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0100000000000001e-07 2024-09-16T11:44:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929317+0000", 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.161e-06 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929320+0000", 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1580000000000001e-06 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929322+0000", 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9929999999999998e-06 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961619+0000", 2024-09-16T11:44:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032296392 2024-09-16T11:44:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961659+0000", 2024-09-16T11:44:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9805000000000003e-05 2024-09-16T11:44:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961694+0000", 2024-09-16T11:44:41.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5182999999999998e-05 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89644 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4243456~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7987432510000003, 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032467912000000002, 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.778 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89644 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0100000000000001e-07 2024-09-16T11:44:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929325+0000", 2024-09-16T11:44:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9899999999999999e-07 2024-09-16T11:44:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929328+0000", 2024-09-16T11:44:41.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7539999999999998e-06 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929330+0000", 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3690000000000001e-06 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961725+0000", 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032394088000000001 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961759+0000", 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4316999999999997e-05 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961792+0000", 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3284000000000002e-05 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89645 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4595712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7987351790000004, 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032554313000000001, 2024-09-16T11:44:41.782 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89645 2024-09-16T11:44:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.99e-07 2024-09-16T11:44:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929333+0000", 2024-09-16T11:44:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9299999999999996e-07 2024-09-16T11:44:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929336+0000", 2024-09-16T11:44:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8459999999999999e-06 2024-09-16T11:44:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929340+0000", 2024-09-16T11:44:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6370000000000001e-06 2024-09-16T11:44:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961814+0000", 2024-09-16T11:44:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032474461000000003 2024-09-16T11:44:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961852+0000", 2024-09-16T11:44:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7877999999999999e-05 2024-09-16T11:44:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961887+0000", 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4399e-05 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89646 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4333568~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7987255419999997, 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032641146000000003, 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.787 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89646 2024-09-16T11:44:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:41.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.04e-07 2024-09-16T11:44:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929343+0000", 2024-09-16T11:44:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7000000000000003e-07 2024-09-16T11:44:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929346+0000", 2024-09-16T11:44:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7350000000000001e-06 2024-09-16T11:44:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929349+0000", 2024-09-16T11:44:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7030000000000002e-06 2024-09-16T11:44:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961916+0000", 2024-09-16T11:44:41.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032567634999999998 2024-09-16T11:44:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961950+0000", 2024-09-16T11:44:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3510000000000003e-05 2024-09-16T11:44:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961983+0000", 2024-09-16T11:44:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3389000000000002e-05 2024-09-16T11:44:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.791 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89647 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4685824~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:44:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7987171399999999, 2024-09-16T11:44:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032727342, 2024-09-16T11:44:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89647 2024-09-16T11:44:41.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:44:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:44:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931111+0000", 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001760462 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931114+0000", 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.67e-06 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931122+0000", 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6059999999999997e-06 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931126+0000", 2024-09-16T11:44:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-06 2024-09-16T11:44:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962005+0000", 2024-09-16T11:44:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.030879125 2024-09-16T11:44:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962044+0000", 2024-09-16T11:44:41.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8779000000000003e-05 2024-09-16T11:44:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962078+0000", 2024-09-16T11:44:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4010000000000001e-05 2024-09-16T11:44:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89656 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4784128~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.796 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7968670910000002, 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031742976999999999, 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89656 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:44:41.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:44:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931201+0000", 2024-09-16T11:44:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0699999999999999e-07 2024-09-16T11:44:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931202+0000", 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5900000000000005e-07 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931204+0000", 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8439999999999998e-06 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931206+0000", 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.871e-06 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962877+0000", 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031670831000000003 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962910+0000", 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3405999999999998e-05 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962943+0000", 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2858999999999998e-05 2024-09-16T11:44:41.800 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89657 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5136384~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7968592799999996, 2024-09-16T11:44:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031827938, 2024-09-16T11:44:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.801 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89657 2024-09-16T11:44:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.28e-07 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931209+0000", 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9099999999999991e-07 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931213+0000", 2024-09-16T11:44:41.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7639999999999999e-06 2024-09-16T11:44:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931217+0000", 2024-09-16T11:44:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4759999999999998e-06 2024-09-16T11:44:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962964+0000", 2024-09-16T11:44:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031747598000000002 2024-09-16T11:44:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963003+0000", 2024-09-16T11:44:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8287000000000001e-05 2024-09-16T11:44:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963036+0000", 2024-09-16T11:44:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3593999999999997e-05 2024-09-16T11:44:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89658 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4874240~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7968317850000002, 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031905067000000002, 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89658 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4700000000000001e-07 2024-09-16T11:44:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931238+0000", 2024-09-16T11:44:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:44:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931245+0000", 2024-09-16T11:44:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.64e-06 2024-09-16T11:44:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931248+0000", 2024-09-16T11:44:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5719999999999999e-06 2024-09-16T11:44:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963066+0000", 2024-09-16T11:44:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031817326999999999 2024-09-16T11:44:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963107+0000", 2024-09-16T11:44:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1220000000000002e-05 2024-09-16T11:44:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963141+0000", 2024-09-16T11:44:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3896999999999997e-05 2024-09-16T11:44:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89659 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4964352~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7968154829999996, 2024-09-16T11:44:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031984460999999999, 2024-09-16T11:44:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89659 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931254+0000", 2024-09-16T11:44:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.776e-06 2024-09-16T11:44:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931262+0000", 2024-09-16T11:44:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1680000000000004e-06 2024-09-16T11:44:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931266+0000", 2024-09-16T11:44:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.732e-06 2024-09-16T11:44:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963165+0000", 2024-09-16T11:44:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031898693999999998 2024-09-16T11:44:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963202+0000", 2024-09-16T11:44:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7517e-05 2024-09-16T11:44:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963236+0000", 2024-09-16T11:44:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4230000000000003e-05 2024-09-16T11:44:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89660 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5226496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7967974629999999, 2024-09-16T11:44:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032062368000000001, 2024-09-16T11:44:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89660 2024-09-16T11:44:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.39e-07 2024-09-16T11:44:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931272+0000", 2024-09-16T11:44:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:44:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931279+0000", 2024-09-16T11:44:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931282+0000", 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.721e-06 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963266+0000", 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031983971999999999 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963300+0000", 2024-09-16T11:44:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3318000000000002e-05 2024-09-16T11:44:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963332+0000", 2024-09-16T11:44:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2768000000000002e-05 2024-09-16T11:44:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89661 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5054464~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:44:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7967810389999999, 2024-09-16T11:44:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032139394000000002, 2024-09-16T11:44:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89661 2024-09-16T11:44:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:44:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:44:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933104+0000", 2024-09-16T11:44:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00181734 2024-09-16T11:44:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933107+0000", 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5140000000000002e-06 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933115+0000", 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933124+0000", 2024-09-16T11:44:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8910000000000001e-06 2024-09-16T11:44:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963354+0000", 2024-09-16T11:44:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.030230825999999999 2024-09-16T11:44:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963392+0000", 2024-09-16T11:44:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7765000000000002e-05 2024-09-16T11:44:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963426+0000", 2024-09-16T11:44:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3603000000000003e-05 2024-09-16T11:44:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:41.824 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:41.826 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:41.826 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:41.827 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:41.827 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:41.823+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:41.926 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:41.924+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:41.966 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:41.966 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:41.967 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:41.967 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:41.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:42.027 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:42.025+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:42.048 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:44:42.048 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:44:42.048 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-16T11:44:42.049 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:44:42.049 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:44:42.049 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-16T11:44:42.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:44:42.082 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:42.082 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:42.083 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:42.083 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:42.083 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:42.093 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:42.127 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:42.126+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:42.160 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:42.160 INFO:teuthology.orchestra.run.smithi192.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T11:44:42.160 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:42.228 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:42.226+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:42.239 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:42.239 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:42.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:42.239 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:42.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.239 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19364 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1171456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.239 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:42.239 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.722277130999998, 2024-09-16T11:44:42.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049001805000000002, 2024-09-16T11:44:42.239 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.239 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19364 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:42.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513280+0000", 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7090000000000001e-06 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513286+0000", 2024-09-16T11:44:42.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5840000000000002e-06 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513290+0000", 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3680000000000004e-06 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562121+0000", 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048831324000000002 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562195+0000", 2024-09-16T11:44:42.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3415000000000001e-05 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562280+0000", 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4975999999999994e-05 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19365 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1490944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.722257392000003, 2024-09-16T11:44:42.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049335349000000001, 2024-09-16T11:44:42.244 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.244 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.244 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.244 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.244 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.244 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19365 2024-09-16T11:44:42.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.244 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:42.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513300+0000", 2024-09-16T11:44:42.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3020000000000002e-06 2024-09-16T11:44:42.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513306+0000", 2024-09-16T11:44:42.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8780000000000003e-06 2024-09-16T11:44:42.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513310+0000", 2024-09-16T11:44:42.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1309999999999999e-06 2024-09-16T11:44:42.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.247 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562322+0000", 2024-09-16T11:44:42.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049011635999999997 2024-09-16T11:44:42.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.247 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562544+0000", 2024-09-16T11:44:42.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022168000000000001 2024-09-16T11:44:42.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.248 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.248 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562633+0000", 2024-09-16T11:44:42.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9350000000000003e-05 2024-09-16T11:44:42.248 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.248 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.248 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.248 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19366 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1916928~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.248 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.722240460000002, 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049865073000000003, 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19366 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:42.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:42.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515102+0000", 2024-09-16T11:44:42.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017877419999999999 2024-09-16T11:44:42.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515106+0000", 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4450000000000001e-06 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515113+0000", 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7680000000000008e-06 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515117+0000", 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562756+0000", 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047638579 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563093+0000", 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000337516 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563180+0000", 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6420000000000003e-05 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.252 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19371 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3194880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.720343253000003, 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048948356999999998, 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19371 2024-09-16T11:44:42.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.254 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:42.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:42.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515214+0000", 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5969999999999999e-06 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515219+0000", 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4559999999999999e-06 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515223+0000", 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563991+0000", 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048768911999999998 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564063+0000", 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2010999999999999e-05 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564160+0000", 2024-09-16T11:44:42.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6617e-05 2024-09-16T11:44:42.257 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.257 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.257 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.257 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19372 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3620864~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.257 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:42.257 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.720329036000003, 2024-09-16T11:44:42.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049130502, 2024-09-16T11:44:42.257 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.257 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19372 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515228+0000", 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515233+0000", 2024-09-16T11:44:42.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515237+0000", 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.27e-06 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564199+0000", 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048961787999999999 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564275+0000", 2024-09-16T11:44:42.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6001000000000006e-05 2024-09-16T11:44:42.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564356+0000", 2024-09-16T11:44:42.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1941999999999996e-05 2024-09-16T11:44:42.261 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.261 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.261 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.261 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19373 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3940352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.261 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:42.261 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.720314635999998, 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049289588000000002, 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19373 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515241+0000", 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515242+0000", 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.528e-06 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515248+0000", 2024-09-16T11:44:42.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:42.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515251+0000", 2024-09-16T11:44:42.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2480000000000001e-06 2024-09-16T11:44:42.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564388+0000", 2024-09-16T11:44:42.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049137313000000002 2024-09-16T11:44:42.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564455+0000", 2024-09-16T11:44:42.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7049000000000001e-05 2024-09-16T11:44:42.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564530+0000", 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4626e-05 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19374 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 4046848~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.720300233000003, 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049457133, 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.266 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.267 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.267 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19374 2024-09-16T11:44:42.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.267 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:42.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:42.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:42.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:42.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515257+0000", 2024-09-16T11:44:42.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5740000000000001e-06 2024-09-16T11:44:42.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515262+0000", 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2399999999999998e-06 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515265+0000", 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3909999999999998e-06 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564555+0000", 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049290002999999999 2024-09-16T11:44:42.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564628+0000", 2024-09-16T11:44:42.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2559999999999996e-05 2024-09-16T11:44:42.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564712+0000", 2024-09-16T11:44:42.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4040000000000002e-05 2024-09-16T11:44:42.270 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19375 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3301376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.720286090000002, 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049606631999999998, 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.271 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19375 2024-09-16T11:44:42.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.272 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:42.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:42.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:42.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:44:42.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515271+0000", 2024-09-16T11:44:42.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5230000000000001e-06 2024-09-16T11:44:42.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515276+0000", 2024-09-16T11:44:42.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4650000000000002e-06 2024-09-16T11:44:42.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515279+0000", 2024-09-16T11:44:42.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0630000000000002e-06 2024-09-16T11:44:42.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564738+0000", 2024-09-16T11:44:42.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049459001000000002 2024-09-16T11:44:42.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564780+0000", 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1693999999999998e-05 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564876+0000", 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5571999999999999e-05 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.275 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19376 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3727360~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.720272178000002, 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049720590000000002, 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19376 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:42.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517103+0000", 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001820276 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517108+0000", 2024-09-16T11:44:42.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5410000000000002e-06 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517120+0000", 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2092e-05 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517125+0000", 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6829999999999996e-06 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564870+0000", 2024-09-16T11:44:42.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047745040000000002 2024-09-16T11:44:42.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564921+0000", 2024-09-16T11:44:42.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0538999999999998e-05 2024-09-16T11:44:42.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565003+0000", 2024-09-16T11:44:42.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2418999999999998e-05 2024-09-16T11:44:42.280 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.280 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.280 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.280 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19492 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 434176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.280 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:42.280 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.718343322000003, 2024-09-16T11:44:42.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048926948999999997, 2024-09-16T11:44:42.280 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.280 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.280 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19492 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:42.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3099999999999999e-07 2024-09-16T11:44:42.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517213+0000", 2024-09-16T11:44:42.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6599999999999994e-07 2024-09-16T11:44:42.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517216+0000", 2024-09-16T11:44:42.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.706e-06 2024-09-16T11:44:42.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517218+0000", 2024-09-16T11:44:42.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.035e-06 2024-09-16T11:44:42.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565959+0000", 2024-09-16T11:44:42.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048741380000000001 2024-09-16T11:44:42.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566036+0000", 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7362999999999998e-05 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566139+0000", 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102268 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19493 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 860160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.718321650999997, 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049086604999999998, 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19493 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517234+0000", 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.37e-07 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517236+0000", 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7150000000000001e-06 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517243+0000", 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9489999999999999e-06 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517246+0000", 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.636e-06 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566163+0000", 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048917284999999998 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566237+0000", 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3196999999999997e-05 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566320+0000", 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3286000000000002e-05 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.288 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19494 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1179648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.718301035000003, 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049259562999999999, 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19494 2024-09-16T11:44:42.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:42.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517256+0000", 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.888e-06 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517265+0000", 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8629999999999997e-06 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517269+0000", 2024-09-16T11:44:42.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2039999999999999e-06 2024-09-16T11:44:42.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566352+0000", 2024-09-16T11:44:42.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049083107000000001 2024-09-16T11:44:42.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566429+0000", 2024-09-16T11:44:42.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7025000000000002e-05 2024-09-16T11:44:42.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.293 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566514+0000", 2024-09-16T11:44:42.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4121000000000003e-05 2024-09-16T11:44:42.293 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.293 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.293 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.293 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19495 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1286144~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.293 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:42.293 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.718281611999998, 2024-09-16T11:44:42.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049492880000000003, 2024-09-16T11:44:42.293 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.294 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.294 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.294 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.294 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.294 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19495 2024-09-16T11:44:42.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.294 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:42.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:42.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517274+0000", 2024-09-16T11:44:42.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:44:42.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517276+0000", 2024-09-16T11:44:42.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8950000000000001e-06 2024-09-16T11:44:42.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517281+0000", 2024-09-16T11:44:42.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3879999999999999e-06 2024-09-16T11:44:42.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517285+0000", 2024-09-16T11:44:42.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5930000000000002e-06 2024-09-16T11:44:42.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566579+0000", 2024-09-16T11:44:42.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049294175000000003 2024-09-16T11:44:42.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566670+0000", 2024-09-16T11:44:42.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1096999999999999e-05 2024-09-16T11:44:42.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566766+0000", 2024-09-16T11:44:42.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6355999999999997e-05 2024-09-16T11:44:42.297 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19496 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1605632~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.718266526999997, 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049632981999999999, 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.298 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19496 2024-09-16T11:44:42.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.299 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:42.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:42.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517289+0000", 2024-09-16T11:44:42.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:42.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517290+0000", 2024-09-16T11:44:42.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.581e-06 2024-09-16T11:44:42.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517300+0000", 2024-09-16T11:44:42.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:42.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517304+0000", 2024-09-16T11:44:42.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:44:42.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566789+0000", 2024-09-16T11:44:42.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049485102000000003 2024-09-16T11:44:42.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566843+0000", 2024-09-16T11:44:42.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3937000000000002e-05 2024-09-16T11:44:42.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566921+0000", 2024-09-16T11:44:42.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8397000000000002e-05 2024-09-16T11:44:42.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.302 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.303 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19497 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 114688~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.303 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:42.303 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.718246831999998, 2024-09-16T11:44:42.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049761367000000001, 2024-09-16T11:44:42.303 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.303 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.303 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19497 2024-09-16T11:44:42.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.303 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:42.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:42.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519116+0000", 2024-09-16T11:44:42.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018079439999999999 2024-09-16T11:44:42.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519120+0000", 2024-09-16T11:44:42.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2760000000000002e-06 2024-09-16T11:44:42.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519134+0000", 2024-09-16T11:44:42.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3468999999999999e-05 2024-09-16T11:44:42.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519140+0000", 2024-09-16T11:44:42.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.516e-06 2024-09-16T11:44:42.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566943+0000", 2024-09-16T11:44:42.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047802561 2024-09-16T11:44:42.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566982+0000", 2024-09-16T11:44:42.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9269e-05 2024-09-16T11:44:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567070+0000", 2024-09-16T11:44:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7331999999999997e-05 2024-09-16T11:44:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19504 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 966656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.716318344999998, 2024-09-16T11:44:42.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049045863000000002, 2024-09-16T11:44:42.308 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.308 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.308 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.308 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.308 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.308 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19504 2024-09-16T11:44:42.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.308 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:42.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519239+0000", 2024-09-16T11:44:42.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.962e-06 2024-09-16T11:44:42.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519244+0000", 2024-09-16T11:44:42.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5480000000000001e-06 2024-09-16T11:44:42.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519248+0000", 2024-09-16T11:44:42.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.641e-06 2024-09-16T11:44:42.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568150+0000", 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048901864000000003 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568208+0000", 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8399000000000001e-05 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568283+0000", 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4105999999999998e-05 2024-09-16T11:44:42.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19505 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1392640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.71630304, 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049202304000000002, 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.312 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19505 2024-09-16T11:44:42.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.313 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:42.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:42.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:42.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:44:42.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519254+0000", 2024-09-16T11:44:42.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.761e-06 2024-09-16T11:44:42.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519257+0000", 2024-09-16T11:44:42.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.343e-06 2024-09-16T11:44:42.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519259+0000", 2024-09-16T11:44:42.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0559999999999999e-06 2024-09-16T11:44:42.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568307+0000", 2024-09-16T11:44:42.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049047445000000002 2024-09-16T11:44:42.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568382+0000", 2024-09-16T11:44:42.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5574999999999994e-05 2024-09-16T11:44:42.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568454+0000", 2024-09-16T11:44:42.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1799999999999997e-05 2024-09-16T11:44:42.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.316 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.317 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19506 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1712128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.317 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:42.317 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.716293002, 2024-09-16T11:44:42.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049341532, 2024-09-16T11:44:42.317 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.317 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.317 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.317 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19506 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521102+0000", 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018404700000000001 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521106+0000", 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8820000000000003e-06 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521119+0000", 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2546e-05 2024-09-16T11:44:42.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521125+0000", 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0249999999999999e-06 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568484+0000", 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047359120999999997 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568542+0000", 2024-09-16T11:44:42.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8360999999999999e-05 2024-09-16T11:44:42.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568604+0000", 2024-09-16T11:44:42.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1126999999999999e-05 2024-09-16T11:44:42.321 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.321 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.321 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.321 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19515 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1499136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.321 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:42.321 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 46.714300717, 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048949431000000002, 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19515 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523130+0000", 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00187604 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523134+0000", 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:44:42.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523144+0000", 2024-09-16T11:44:42.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2010000000000005e-06 2024-09-16T11:44:42.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523149+0000", 2024-09-16T11:44:42.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0039999999999999e-06 2024-09-16T11:44:42.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570113+0000", 2024-09-16T11:44:42.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.046964517999999997 2024-09-16T11:44:42.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570157+0000", 2024-09-16T11:44:42.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4056e-05 2024-09-16T11:44:42.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570204+0000", 2024-09-16T11:44:42.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6591999999999997e-05 2024-09-16T11:44:42.326 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.326 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.326 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.326 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:44111 4.8 4:17f3f47e:::rbd_data.11c1e170f5f0.000000000000002f:head [delete] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:44:42.326 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:42.326 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 32.252320435999998, 2024-09-16T11:44:42.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93903426400000001, 2024-09-16T11:44:42.326 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.326 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:42.326 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.327 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.327 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.327 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 44111 2024-09-16T11:44:42.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.327 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:42.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7199999999999998e-07 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983239+0000", 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9879999999999998e-06 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983248+0000", 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.371e-06 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983253+0000", 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1150000000000002e-06 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983303+0000", 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0380000000000002e-05 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983311+0000", 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7519999999999996e-06 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920618+0000", 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93730672000000004 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920730+0000", 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011265900000000001 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:42.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920842+0000", 2024-09-16T11:44:42.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111638 2024-09-16T11:44:42.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:42.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.921503+0000", 2024-09-16T11:44:42.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066074299999999999 2024-09-16T11:44:42.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:42.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922208+0000", 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00070492899999999997 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922233+0000", 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5922999999999999e-05 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922269+0000", 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5373999999999997e-05 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.332 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.333 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.333 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:42.334 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:42.335 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:42.332+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:42.435 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:42.433+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:42.472 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:42.473 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:42.473 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:42.473 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:42.481 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:42.536 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:42.534+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:42.619 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:42.619 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:42.620 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:42.620 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:42.620 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:42.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:42.671 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:42.669+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:42.727 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:42.727 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16853 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3375104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.377228950999999, 2024-09-16T11:44:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027791907000000001, 2024-09-16T11:44:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16853 2024-09-16T11:44:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347568+0000", 2024-09-16T11:44:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:44:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347570+0000", 2024-09-16T11:44:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:44:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347588+0000", 2024-09-16T11:44:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7914000000000002e-05 2024-09-16T11:44:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347592+0000", 2024-09-16T11:44:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3150000000000002e-06 2024-09-16T11:44:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375199+0000", 2024-09-16T11:44:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027606065999999999 2024-09-16T11:44:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375276+0000", 2024-09-16T11:44:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7527000000000005e-05 2024-09-16T11:44:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375359+0000", 2024-09-16T11:44:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3238000000000005e-05 2024-09-16T11:44:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16854 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.377199253999997, 2024-09-16T11:44:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027945148999999999, 2024-09-16T11:44:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16854 2024-09-16T11:44:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347598+0000", 2024-09-16T11:44:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:44:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347600+0000", 2024-09-16T11:44:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:44:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347606+0000", 2024-09-16T11:44:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0580000000000003e-06 2024-09-16T11:44:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347609+0000", 2024-09-16T11:44:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:44:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375386+0000", 2024-09-16T11:44:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027776809 2024-09-16T11:44:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375461+0000", 2024-09-16T11:44:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4868000000000002e-05 2024-09-16T11:44:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375542+0000", 2024-09-16T11:44:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1253000000000003e-05 2024-09-16T11:44:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16855 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 4120576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.377182914999999, 2024-09-16T11:44:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028111094999999999, 2024-09-16T11:44:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16855 2024-09-16T11:44:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347614+0000", 2024-09-16T11:44:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:44:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347616+0000", 2024-09-16T11:44:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:44:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347623+0000", 2024-09-16T11:44:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0099999999999998e-06 2024-09-16T11:44:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347626+0000", 2024-09-16T11:44:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2140000000000001e-06 2024-09-16T11:44:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375569+0000", 2024-09-16T11:44:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027942649999999999 2024-09-16T11:44:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375643+0000", 2024-09-16T11:44:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3949999999999995e-05 2024-09-16T11:44:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375725+0000", 2024-09-16T11:44:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1992999999999999e-05 2024-09-16T11:44:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16856 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3801088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.377160844000002, 2024-09-16T11:44:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028268272000000001, 2024-09-16T11:44:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16856 2024-09-16T11:44:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347636+0000", 2024-09-16T11:44:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:44:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347638+0000", 2024-09-16T11:44:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.86e-06 2024-09-16T11:44:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347644+0000", 2024-09-16T11:44:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1739999999999997e-06 2024-09-16T11:44:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347656+0000", 2024-09-16T11:44:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2198000000000001e-05 2024-09-16T11:44:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375751+0000", 2024-09-16T11:44:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028094507000000001 2024-09-16T11:44:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375824+0000", 2024-09-16T11:44:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3178999999999998e-05 2024-09-16T11:44:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375904+0000", 2024-09-16T11:44:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9985000000000001e-05 2024-09-16T11:44:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16857 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.377134341999998, 2024-09-16T11:44:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028275403000000001, 2024-09-16T11:44:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16857 2024-09-16T11:44:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.46e-07 2024-09-16T11:44:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347666+0000", 2024-09-16T11:44:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8739999999999997e-06 2024-09-16T11:44:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347672+0000", 2024-09-16T11:44:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1059999999999996e-06 2024-09-16T11:44:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347676+0000", 2024-09-16T11:44:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:44:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375930+0000", 2024-09-16T11:44:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028254042 2024-09-16T11:44:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375937+0000", 2024-09-16T11:44:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5519999999999999e-06 2024-09-16T11:44:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16858 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.377116696999998, 2024-09-16T11:44:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028284592000000001, 2024-09-16T11:44:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16858 2024-09-16T11:44:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:44:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347683+0000", 2024-09-16T11:44:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6809999999999998e-06 2024-09-16T11:44:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347689+0000", 2024-09-16T11:44:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347695+0000", 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0020000000000003e-06 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375958+0000", 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028263870999999999 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375964+0000", 2024-09-16T11:44:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8209999999999998e-06 2024-09-16T11:44:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16992 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.375542009, 2024-09-16T11:44:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027704776, 2024-09-16T11:44:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16992 2024-09-16T11:44:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349255+0000", 2024-09-16T11:44:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:44:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349260+0000", 2024-09-16T11:44:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5589999999999996e-06 2024-09-16T11:44:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349267+0000", 2024-09-16T11:44:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4019999999999999e-06 2024-09-16T11:44:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349270+0000", 2024-09-16T11:44:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6780000000000002e-06 2024-09-16T11:44:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376953+0000", 2024-09-16T11:44:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682933 2024-09-16T11:44:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376959+0000", 2024-09-16T11:44:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:44:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16993 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.375520684000001, 2024-09-16T11:44:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027707908999999999, 2024-09-16T11:44:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16993 2024-09-16T11:44:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349278+0000", 2024-09-16T11:44:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.629e-06 2024-09-16T11:44:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349291+0000", 2024-09-16T11:44:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614e-05 2024-09-16T11:44:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349295+0000", 2024-09-16T11:44:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1189999999999999e-06 2024-09-16T11:44:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376978+0000", 2024-09-16T11:44:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682445 2024-09-16T11:44:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376984+0000", 2024-09-16T11:44:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7590000000000003e-06 2024-09-16T11:44:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16994 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.375496673000001, 2024-09-16T11:44:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027708753999999999, 2024-09-16T11:44:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16994 2024-09-16T11:44:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:44:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349302+0000", 2024-09-16T11:44:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.722e-06 2024-09-16T11:44:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349308+0000", 2024-09-16T11:44:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:44:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349312+0000", 2024-09-16T11:44:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5310000000000002e-06 2024-09-16T11:44:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377003+0000", 2024-09-16T11:44:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027691308000000001 2024-09-16T11:44:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377008+0000", 2024-09-16T11:44:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4550000000000003e-06 2024-09-16T11:44:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16999 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.375353795000002, 2024-09-16T11:44:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027705270000000001, 2024-09-16T11:44:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16999 2024-09-16T11:44:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.46e-07 2024-09-16T11:44:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349445+0000", 2024-09-16T11:44:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.637e-06 2024-09-16T11:44:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349451+0000", 2024-09-16T11:44:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9959999999999999e-06 2024-09-16T11:44:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349454+0000", 2024-09-16T11:44:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.647e-06 2024-09-16T11:44:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377142+0000", 2024-09-16T11:44:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027687942 2024-09-16T11:44:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377148+0000", 2024-09-16T11:44:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7019999999999998e-06 2024-09-16T11:44:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17000 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.375338288000002, 2024-09-16T11:44:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.02771881, 2024-09-16T11:44:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17000 2024-09-16T11:44:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0600000000000001e-07 2024-09-16T11:44:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349460+0000", 2024-09-16T11:44:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.736e-06 2024-09-16T11:44:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349466+0000", 2024-09-16T11:44:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2419999999999997e-06 2024-09-16T11:44:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349470+0000", 2024-09-16T11:44:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5480000000000002e-06 2024-09-16T11:44:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377171+0000", 2024-09-16T11:44:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027701282000000001 2024-09-16T11:44:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377177+0000", 2024-09-16T11:44:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5960000000000003e-06 2024-09-16T11:44:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17001 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.375322537999999, 2024-09-16T11:44:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027739243, 2024-09-16T11:44:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17001 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:44:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349476+0000", 2024-09-16T11:44:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T11:44:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349482+0000", 2024-09-16T11:44:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2389999999999999e-06 2024-09-16T11:44:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349486+0000", 2024-09-16T11:44:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5980000000000001e-06 2024-09-16T11:44:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377206+0000", 2024-09-16T11:44:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027720037999999999 2024-09-16T11:44:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377213+0000", 2024-09-16T11:44:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3440000000000003e-06 2024-09-16T11:44:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17433 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 5898240~4888] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.339358883999999, 2024-09-16T11:44:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027709398, 2024-09-16T11:44:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17433 2024-09-16T11:44:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385438+0000", 2024-09-16T11:44:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385439+0000", 2024-09-16T11:44:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5120000000000001e-06 2024-09-16T11:44:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385445+0000", 2024-09-16T11:44:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4e-06 2024-09-16T11:44:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385448+0000", 2024-09-16T11:44:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1669999999999999e-06 2024-09-16T11:44:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412916+0000", 2024-09-16T11:44:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027468137 2024-09-16T11:44:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412988+0000", 2024-09-16T11:44:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1606000000000005e-05 2024-09-16T11:44:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413147+0000", 2024-09-16T11:44:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015924299999999999 2024-09-16T11:44:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17434 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6004736~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.339344294, 2024-09-16T11:44:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027777461, 2024-09-16T11:44:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17434 2024-09-16T11:44:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385454+0000", 2024-09-16T11:44:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6729999999999999e-06 2024-09-16T11:44:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385459+0000", 2024-09-16T11:44:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4079999999999997e-06 2024-09-16T11:44:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385463+0000", 2024-09-16T11:44:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.269e-06 2024-09-16T11:44:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413157+0000", 2024-09-16T11:44:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027694611000000001 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413193+0000", 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5673999999999998e-05 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413230+0000", 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6454e-05 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17435 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6324224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.339330140999998, 2024-09-16T11:44:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929059999999999, 2024-09-16T11:44:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17435 2024-09-16T11:44:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385467+0000", 2024-09-16T11:44:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385468+0000", 2024-09-16T11:44:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6300000000000001e-06 2024-09-16T11:44:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385474+0000", 2024-09-16T11:44:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1299999999999998e-06 2024-09-16T11:44:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385478+0000", 2024-09-16T11:44:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2880000000000001e-06 2024-09-16T11:44:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413257+0000", 2024-09-16T11:44:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027779608000000001 2024-09-16T11:44:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413331+0000", 2024-09-16T11:44:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3639000000000004e-05 2024-09-16T11:44:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413395+0000", 2024-09-16T11:44:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4424999999999994e-05 2024-09-16T11:44:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17436 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7176192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.339314479000002, 2024-09-16T11:44:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028016718999999999, 2024-09-16T11:44:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17436 2024-09-16T11:44:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385484+0000", 2024-09-16T11:44:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5179999999999999e-06 2024-09-16T11:44:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385489+0000", 2024-09-16T11:44:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7049999999999996e-06 2024-09-16T11:44:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385493+0000", 2024-09-16T11:44:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T11:44:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413422+0000", 2024-09-16T11:44:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929611999999999 2024-09-16T11:44:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413462+0000", 2024-09-16T11:44:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9585000000000001e-05 2024-09-16T11:44:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413499+0000", 2024-09-16T11:44:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6644999999999999e-05 2024-09-16T11:44:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17437 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6750208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.339300064, 2024-09-16T11:44:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028167168999999999, 2024-09-16T11:44:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17437 2024-09-16T11:44:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385497+0000", 2024-09-16T11:44:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:44:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385498+0000", 2024-09-16T11:44:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.592e-06 2024-09-16T11:44:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385504+0000", 2024-09-16T11:44:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4090000000000002e-06 2024-09-16T11:44:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385513+0000", 2024-09-16T11:44:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4460000000000007e-06 2024-09-16T11:44:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413526+0000", 2024-09-16T11:44:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028012866000000001 2024-09-16T11:44:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413601+0000", 2024-09-16T11:44:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5388000000000004e-05 2024-09-16T11:44:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413663+0000", 2024-09-16T11:44:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.211e-05 2024-09-16T11:44:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17438 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7495680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.339277305000003, 2024-09-16T11:44:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028224439, 2024-09-16T11:44:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17438 2024-09-16T11:44:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4900000000000001e-07 2024-09-16T11:44:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385521+0000", 2024-09-16T11:44:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:44:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385526+0000", 2024-09-16T11:44:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2680000000000002e-06 2024-09-16T11:44:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385529+0000", 2024-09-16T11:44:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2040000000000002e-06 2024-09-16T11:44:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413674+0000", 2024-09-16T11:44:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028144226000000001 2024-09-16T11:44:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413710+0000", 2024-09-16T11:44:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6665000000000002e-05 2024-09-16T11:44:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413743+0000", 2024-09-16T11:44:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3155999999999999e-05 2024-09-16T11:44:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17439 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7921664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.339216151000002, 2024-09-16T11:44:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028326131000000001, 2024-09-16T11:44:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17439 2024-09-16T11:44:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385581+0000", 2024-09-16T11:44:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9299999999999998e-07 2024-09-16T11:44:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385583+0000", 2024-09-16T11:44:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6220000000000001e-06 2024-09-16T11:44:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385592+0000", 2024-09-16T11:44:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5220000000000007e-06 2024-09-16T11:44:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385595+0000", 2024-09-16T11:44:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8669999999999998e-06 2024-09-16T11:44:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413771+0000", 2024-09-16T11:44:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028175875999999999 2024-09-16T11:44:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413845+0000", 2024-09-16T11:44:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4382999999999996e-05 2024-09-16T11:44:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413906+0000", 2024-09-16T11:44:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1367999999999999e-05 2024-09-16T11:44:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17440 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.339198943, 2024-09-16T11:44:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028391117, 2024-09-16T11:44:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17440 2024-09-16T11:44:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385598+0000", 2024-09-16T11:44:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:44:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385603+0000", 2024-09-16T11:44:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:44:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385607+0000", 2024-09-16T11:44:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4869999999999998e-06 2024-09-16T11:44:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385609+0000", 2024-09-16T11:44:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2759999999999999e-06 2024-09-16T11:44:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413916+0000", 2024-09-16T11:44:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028307253000000001 2024-09-16T11:44:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413955+0000", 2024-09-16T11:44:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8652000000000002e-05 2024-09-16T11:44:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413989+0000", 2024-09-16T11:44:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3587999999999998e-05 2024-09-16T11:44:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.831 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.831 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:42.831 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:42.831 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:42.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:42.833 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:42.831+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:42.933 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:42.931+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:42.971 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:42.972 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:42.972 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:42.972 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:42.980 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:43.033 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:43.031+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:43.055 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T11:44:43.087 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:43.087 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:43.088 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:43.088 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:43.088 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:43.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:43.134 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:43.132+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:43.234 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:43.232+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:43.236 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:43.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:43.236 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:43.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.236 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19545 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4415488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.236 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:43.236 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.709602115000003, 2024-09-16T11:44:43.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043429581000000002, 2024-09-16T11:44:43.236 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.237 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.237 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.237 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.237 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.237 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19545 2024-09-16T11:44:43.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.237 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:43.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:43.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:43.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:43.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523839+0000", 2024-09-16T11:44:43.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0220000000000004e-06 2024-09-16T11:44:43.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523845+0000", 2024-09-16T11:44:43.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6500000000000001e-06 2024-09-16T11:44:43.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523848+0000", 2024-09-16T11:44:43.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1879999999999998e-06 2024-09-16T11:44:43.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567092+0000", 2024-09-16T11:44:43.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043244145999999997 2024-09-16T11:44:43.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567176+0000", 2024-09-16T11:44:43.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3825999999999993e-05 2024-09-16T11:44:43.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567263+0000", 2024-09-16T11:44:43.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7423999999999995e-05 2024-09-16T11:44:43.240 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19546 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4734976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.709584274999997, 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043599231000000002, 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.241 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19546 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523954+0000", 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010217099999999999 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523957+0000", 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3270000000000001e-06 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523966+0000", 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6489999999999997e-06 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523971+0000", 2024-09-16T11:44:43.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:43.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567291+0000", 2024-09-16T11:44:43.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043319931999999998 2024-09-16T11:44:43.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567360+0000", 2024-09-16T11:44:43.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8628999999999996e-05 2024-09-16T11:44:43.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567451+0000", 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1012000000000002e-05 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19547 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4841472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.709459418000002, 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043658858000000002, 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.245 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.246 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.246 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.246 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.246 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19547 2024-09-16T11:44:43.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.246 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:43.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:43.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.247 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525102+0000", 2024-09-16T11:44:43.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011257730000000001 2024-09-16T11:44:43.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.247 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.248 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525106+0000", 2024-09-16T11:44:43.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9910000000000004e-06 2024-09-16T11:44:43.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.248 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.248 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525116+0000", 2024-09-16T11:44:43.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0231000000000001e-05 2024-09-16T11:44:43.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.248 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.248 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525137+0000", 2024-09-16T11:44:43.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0828000000000001e-05 2024-09-16T11:44:43.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567478+0000", 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.042340467999999999 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567554+0000", 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6074000000000004e-05 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567635+0000", 2024-09-16T11:44:43.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1493000000000001e-05 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19553 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4947968~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.708232356000003, 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043450397000000002, 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.250 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19553 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1799999999999999e-07 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525205+0000", 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.062e-06 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525208+0000", 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8739999999999999e-06 2024-09-16T11:44:43.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525209+0000", 2024-09-16T11:44:43.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.725e-06 2024-09-16T11:44:43.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568520+0000", 2024-09-16T11:44:43.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043310226 2024-09-16T11:44:43.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568569+0000", 2024-09-16T11:44:43.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8959999999999999e-05 2024-09-16T11:44:43.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568654+0000", 2024-09-16T11:44:43.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5332000000000002e-05 2024-09-16T11:44:43.254 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.254 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.254 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.254 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19554 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5267456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.254 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.708223363999998, 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043659587, 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19554 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527103+0000", 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018907419999999999 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527108+0000", 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6040000000000003e-06 2024-09-16T11:44:43.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527118+0000", 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527122+0000", 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9559999999999996e-06 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568688+0000", 2024-09-16T11:44:43.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041565116999999999 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568768+0000", 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0474000000000002e-05 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568872+0000", 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104028 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19565 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6332416~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.706144792000003, 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043494890000000001, 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19565 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:43.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:43.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527292+0000", 2024-09-16T11:44:43.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.27e-07 2024-09-16T11:44:43.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527293+0000", 2024-09-16T11:44:43.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.917e-06 2024-09-16T11:44:43.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.262 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527299+0000", 2024-09-16T11:44:43.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6699999999999999e-06 2024-09-16T11:44:43.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.262 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527304+0000", 2024-09-16T11:44:43.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6160000000000004e-06 2024-09-16T11:44:43.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570683+0000", 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043379171000000001 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570733+0000", 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0485000000000002e-05 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570786+0000", 2024-09-16T11:44:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2604e-05 2024-09-16T11:44:43.264 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.264 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.264 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.264 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19566 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6758400~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.264 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:43.264 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.706128458000002, 2024-09-16T11:44:43.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043643183000000002, 2024-09-16T11:44:43.264 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.265 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.265 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.265 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.265 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.265 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19566 2024-09-16T11:44:43.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.265 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:43.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:43.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527308+0000", 2024-09-16T11:44:43.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:44:43.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527310+0000", 2024-09-16T11:44:43.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.68e-06 2024-09-16T11:44:43.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527316+0000", 2024-09-16T11:44:43.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9730000000000002e-06 2024-09-16T11:44:43.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527319+0000", 2024-09-16T11:44:43.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.027e-06 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570810+0000", 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043491525000000003 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570882+0000", 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570951+0000", 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8782000000000007e-05 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19567 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.706113752, 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043786291999999997, 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.269 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19567 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529108+0000", 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001785656 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529113+0000", 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1080000000000001e-06 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529122+0000", 2024-09-16T11:44:43.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0550000000000005e-06 2024-09-16T11:44:43.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529127+0000", 2024-09-16T11:44:43.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9849999999999997e-06 2024-09-16T11:44:43.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570997+0000", 2024-09-16T11:44:43.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041869811 2024-09-16T11:44:43.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571063+0000", 2024-09-16T11:44:43.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6340999999999994e-05 2024-09-16T11:44:43.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571108+0000", 2024-09-16T11:44:43.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5336000000000001e-05 2024-09-16T11:44:43.273 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.273 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.273 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.273 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19580 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 8036352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.274 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:43.274 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.704138450999999, 2024-09-16T11:44:43.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043451596000000002, 2024-09-16T11:44:43.274 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.274 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.274 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.274 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.274 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.274 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19580 2024-09-16T11:44:43.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.275 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:43.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:43.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531106+0000", 2024-09-16T11:44:43.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001808179 2024-09-16T11:44:43.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531110+0000", 2024-09-16T11:44:43.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5990000000000004e-06 2024-09-16T11:44:43.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531123+0000", 2024-09-16T11:44:43.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2687000000000001e-05 2024-09-16T11:44:43.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531129+0000", 2024-09-16T11:44:43.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:44:43.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572642+0000", 2024-09-16T11:44:43.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041513474000000002 2024-09-16T11:44:43.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572691+0000", 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8989000000000001e-05 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572749+0000", 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7806999999999997e-05 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.279 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19709 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4849664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.279 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:43.279 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.702078409999999, 2024-09-16T11:44:43.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043537291999999998, 2024-09-16T11:44:43.279 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.279 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.279 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.279 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.279 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.279 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19709 2024-09-16T11:44:43.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.279 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:43.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:43.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531358+0000", 2024-09-16T11:44:43.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:43.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531360+0000", 2024-09-16T11:44:43.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:44:43.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531365+0000", 2024-09-16T11:44:43.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.716e-06 2024-09-16T11:44:43.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531369+0000", 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5410000000000001e-06 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574763+0000", 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394359 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574831+0000", 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7997999999999995e-05 2024-09-16T11:44:43.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574895+0000", 2024-09-16T11:44:43.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3427000000000001e-05 2024-09-16T11:44:43.283 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.283 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.283 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.283 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19710 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4956160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.283 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:43.283 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.702063131000003, 2024-09-16T11:44:43.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043674198999999997, 2024-09-16T11:44:43.284 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.284 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.284 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.284 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.284 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.284 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19710 2024-09-16T11:44:43.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.284 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:43.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:43.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533105+0000", 2024-09-16T11:44:43.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001731961 2024-09-16T11:44:43.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533109+0000", 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3080000000000001e-06 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533120+0000", 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0625000000000001e-05 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533125+0000", 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574923+0000", 2024-09-16T11:44:43.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041797187 2024-09-16T11:44:43.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574991+0000", 2024-09-16T11:44:43.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8752999999999997e-05 2024-09-16T11:44:43.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.575047+0000", 2024-09-16T11:44:43.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5596000000000001e-05 2024-09-16T11:44:43.288 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.288 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.288 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.288 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19719 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5488640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.288 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:43.288 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.700091381999997, 2024-09-16T11:44:43.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043412013999999999, 2024-09-16T11:44:43.288 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.288 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.288 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.288 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.289 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.289 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19719 2024-09-16T11:44:43.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.289 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:43.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533345+0000", 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533347+0000", 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8139999999999999e-06 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533352+0000", 2024-09-16T11:44:43.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4970000000000001e-06 2024-09-16T11:44:43.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533355+0000", 2024-09-16T11:44:43.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.711e-06 2024-09-16T11:44:43.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576629+0000", 2024-09-16T11:44:43.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043274295999999997 2024-09-16T11:44:43.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576706+0000", 2024-09-16T11:44:43.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6711000000000005e-05 2024-09-16T11:44:43.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576757+0000", 2024-09-16T11:44:43.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0509e-05 2024-09-16T11:44:43.292 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.292 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.292 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.293 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19720 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5808128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.293 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:43.293 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.700078732999998, 2024-09-16T11:44:43.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043577006000000001, 2024-09-16T11:44:43.293 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.293 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.293 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.293 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.293 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.293 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19720 2024-09-16T11:44:43.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.293 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:43.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:43.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:43.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.35e-07 2024-09-16T11:44:43.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533359+0000", 2024-09-16T11:44:43.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:44:43.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533364+0000", 2024-09-16T11:44:43.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4349999999999996e-06 2024-09-16T11:44:43.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533367+0000", 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2349999999999999e-06 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576786+0000", 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043418885999999997 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576862+0000", 2024-09-16T11:44:43.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6172000000000002e-05 2024-09-16T11:44:43.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576934+0000", 2024-09-16T11:44:43.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1891999999999995e-05 2024-09-16T11:44:43.297 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.297 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.297 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.297 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19721 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6234112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.297 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:43.297 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.700065213000002, 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043681656999999999, 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19721 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-07 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533373+0000", 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.973e-06 2024-09-16T11:44:43.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533377+0000", 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.403e-06 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533379+0000", 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.058e-06 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576959+0000", 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043579433000000001 2024-09-16T11:44:43.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577015+0000", 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5859000000000002e-05 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577052+0000", 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7620999999999998e-05 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19722 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6340608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.700054485999999, 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043854922999999997, 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19722 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535103+0000", 2024-09-16T11:44:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017217020000000001 2024-09-16T11:44:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535107+0000", 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2819999999999998e-06 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535118+0000", 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0148e-05 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535123+0000", 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:44:43.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577091+0000", 2024-09-16T11:44:43.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041967930000000001 2024-09-16T11:44:43.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577163+0000", 2024-09-16T11:44:43.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1853999999999993e-05 2024-09-16T11:44:43.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577236+0000", 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3699e-05 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19732 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6447104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.698194630000003, 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043399048000000003, 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.306 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19732 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535242+0000", 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535243+0000", 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.793e-06 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535249+0000", 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.643e-06 2024-09-16T11:44:43.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535252+0000", 2024-09-16T11:44:43.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:44:43.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578501+0000", 2024-09-16T11:44:43.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043248343000000002 2024-09-16T11:44:43.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578576+0000", 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5544999999999996e-05 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578640+0000", 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4018999999999998e-05 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19733 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6873088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.310 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:43.311 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.698173789000002, 2024-09-16T11:44:43.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043549681, 2024-09-16T11:44:43.311 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.311 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.311 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.311 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.311 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.311 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19733 2024-09-16T11:44:43.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.311 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:43.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.312 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:43.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535263+0000", 2024-09-16T11:44:43.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:44:43.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535265+0000", 2024-09-16T11:44:43.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9369999999999998e-06 2024-09-16T11:44:43.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535271+0000", 2024-09-16T11:44:43.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9719999999999997e-06 2024-09-16T11:44:43.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535274+0000", 2024-09-16T11:44:43.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:44:43.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578668+0000", 2024-09-16T11:44:43.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043393355000000002 2024-09-16T11:44:43.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578744+0000", 2024-09-16T11:44:43.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6853000000000004e-05 2024-09-16T11:44:43.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578812+0000", 2024-09-16T11:44:43.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.737e-05 2024-09-16T11:44:43.315 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.315 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.315 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.316 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19734 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7299072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.316 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:43.316 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.698158042000003, 2024-09-16T11:44:43.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043699189999999999, 2024-09-16T11:44:43.316 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.316 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.316 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.316 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.316 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.316 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19734 2024-09-16T11:44:43.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.317 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:43.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:43.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537101+0000", 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001823098 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537105+0000", 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537119+0000", 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614999999999999e-05 2024-09-16T11:44:43.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537125+0000", 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578838+0000", 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041712605999999999 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578912+0000", 2024-09-16T11:44:43.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4396999999999999e-05 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578977+0000", 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4858999999999997e-05 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19851 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4431872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 47.696077297999999, 2024-09-16T11:44:43.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043564540999999998, 2024-09-16T11:44:43.321 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.321 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:43.321 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.321 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.321 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.321 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19851 2024-09-16T11:44:43.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.321 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:43.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:43.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539100+0000", 2024-09-16T11:44:43.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017412079999999999 2024-09-16T11:44:43.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539105+0000", 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539116+0000", 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1824999999999999e-05 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539122+0000", 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9859999999999999e-06 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580764+0000", 2024-09-16T11:44:43.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041641299 2024-09-16T11:44:43.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580839+0000", 2024-09-16T11:44:43.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5167999999999996e-05 2024-09-16T11:44:43.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580923+0000", 2024-09-16T11:44:43.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4314999999999995e-05 2024-09-16T11:44:43.324 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55904 4.14 4:2d2dfe38:::rbd_data.11c1b01b799a.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2068993~28159] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.141700397999999, 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.90010917700000004, 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55904 2024-09-16T11:44:43.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.326 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:43.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:43.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:43.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:43.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:43.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:43.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091737+0000", 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.254e-06 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091759+0000", 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2487999999999999e-05 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091769+0000", 2024-09-16T11:44:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0199e-05 2024-09-16T11:44:43.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091774+0000", 2024-09-16T11:44:43.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:44:43.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091830+0000", 2024-09-16T11:44:43.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6700999999999998e-05 2024-09-16T11:44:43.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:44:43.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091840+0000", 2024-09-16T11:44:43.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0128e-05 2024-09-16T11:44:43.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:43.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990355+0000", 2024-09-16T11:44:43.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.89851413099999999 2024-09-16T11:44:43.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:43.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990509+0000", 2024-09-16T11:44:43.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000154442 2024-09-16T11:44:43.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990626+0000", 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000116928 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991526+0000", 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00089961800000000003 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991793+0000", 2024-09-16T11:44:43.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026779000000000001 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991824+0000", 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0380999999999999e-05 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991845+0000", 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0919000000000001e-05 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.331 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:43.332 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:43.332 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:43.333 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:43.374 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:43.372+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:43.435 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:43.434+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:43.437 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:43.437 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:43.438 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:43.446 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:43.536 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:43.535+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:43.585 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:43.585 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:43.585 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:43.585 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:43.585 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:43.593 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:43.677 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:43.676+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:43.732 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:43.733 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:43.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:43.733 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:43.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.733 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.733 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:43.733 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368707370000003, 2024-09-16T11:44:43.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:44:43.733 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.733 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.734 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.734 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.734 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:44:43.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.734 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:43.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:43.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:43.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:44:43.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:44:43.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.737 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.738 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.738 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:43.738 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368685341000003, 2024-09-16T11:44:43.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:44:43.738 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.738 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.738 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:44:43.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.739 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:43.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:43.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:43.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:44:43.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:44:43.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:44:43.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:44:43.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:44:43.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:44:43.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:44:43.742 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.742 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.742 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.742 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368670576, 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:43.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:43.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:43.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:44:43.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:44:43.746 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.747 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.747 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.747 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.747 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:43.747 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368650508000002, 2024-09-16T11:44:43.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:44:43.747 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.747 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.748 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.748 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:44:43.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.748 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:43.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:43.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:44:43.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:44:43.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:44:43.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:44:43.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:44:43.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:44:43.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:44:43.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:44:43.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:44:43.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:44:43.751 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.751 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.751 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.752 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.752 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:43.752 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368619672000001, 2024-09-16T11:44:43.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:44:43.752 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.752 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.752 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.752 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.752 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.752 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:44:43.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.752 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:44:43.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:44:43.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:44:43.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:44:43.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:44:43.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:44:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:44:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368600151999999, 2024-09-16T11:44:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:44:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.756 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.757 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:44:43.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.757 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:43.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:43.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:43.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:44:43.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:44:43.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:44:43.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:44:43.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:44:43.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:44:43.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:44:43.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:44:43.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:44:43.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:44:43.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:44:43.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:44:43.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368572573000002, 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:44:43.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:44:43.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:44:43.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:44:43.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:44:43.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368553878, 2024-09-16T11:44:43.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:44:43.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.769 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368538868999998, 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:44:43.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:44:43.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:44:43.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:44:43.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:44:43.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:44:43.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:44:43.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:44:43.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:44:43.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:44:43.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:44:43.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:44:43.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.774 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.774 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.774 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:43.774 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368506771, 2024-09-16T11:44:43.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:44:43.774 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.774 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.775 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:44:43.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.775 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:43.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:43.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:44:43.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:44:43.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:44:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:44:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:44:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:44:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:44:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:44:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:44:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:44:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:44:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:44:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:44:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:44:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368483152000003, 2024-09-16T11:44:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:44:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.780 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:44:43.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.780 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:43.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:43.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:43.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:43.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:44:43.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:44:43.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:44:43.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:44:43.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:44:43.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:44:43.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:44:43.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:44:43.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:44:43.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:44:43.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:44:43.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:44:43.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.784 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.784 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.784 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:43.784 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368468423000003, 2024-09-16T11:44:43.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:44:43.784 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.785 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.785 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:44:43.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.785 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:43.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:43.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:43.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:44:43.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:44:43.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:44:43.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:44:43.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:44:43.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:44:43.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:44:43.788 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.789 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.789 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.789 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.789 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:43.789 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368446599000002, 2024-09-16T11:44:43.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:44:43.789 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.789 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:44:43.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:43.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:44:43.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:44:43.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:44:43.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368431948999998, 2024-09-16T11:44:43.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:43.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:44:43.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:44:43.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:44:43.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:44:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:44:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:44:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:44:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:44:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:44:43.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:44:43.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:44:43.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:44:43.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:44:43.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:44:43.797 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.798 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.798 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.798 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.798 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:43.798 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368410376, 2024-09-16T11:44:43.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:44:43.798 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.798 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.799 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:44:43.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.799 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:43.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:43.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:43.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:44:43.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:44:43.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:44:43.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.802 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368390929999997, 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:43.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:43.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:44:43.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:43.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:44:43.806 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368354232999998, 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.807 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:44:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:44:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:44:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:44:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:44:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:44:43.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:44:43.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:44:43.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:44:43.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:44:43.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:44:43.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:44:43.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:44:43.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:44:43.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:44:43.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.812 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.812 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.812 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:43.812 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368332287000001, 2024-09-16T11:44:43.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:44:43.812 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.813 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.813 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:44:43.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.813 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:43.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:43.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:43.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:44:43.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.816 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368310401000002, 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:44:43.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:43.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:44:43.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:44:43.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:44:43.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:44:43.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:44:43.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:44:43.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.368295682000003, 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:43.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:43.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:43.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:43.822 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:44:43.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.822 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:43.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:43.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:43.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:43.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:43.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:43.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:43.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:43.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:43.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:43.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:44:43.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:44:43.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:44:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:44:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:44:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:44:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:44:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:44:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:44:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:44:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:44:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:44:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:43.826 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:43.827 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:43.828 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:44:43.828 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:43.826+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:43.928 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:43.926+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:43.954 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:43.954 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:43.967 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:43.967 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:43.967 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:43.967 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:43.970 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:43.970 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:43.975 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:43.984 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:43.984 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:43.999 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:44.000 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:44.015 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:44.015 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:44.029 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:44.027+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:44.030 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:44.030 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:44.046 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:44.046 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:44.062 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:44.062 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:44.078 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:44.078 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:44.087 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:44.130 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:44.128+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:44.190 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:44.191 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:44.199 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:44.232 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:44.230+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:44.297 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:44.297 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:44.305 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:44.333 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:44.331+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:44.402 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:44.402 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:44.410 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:44.454 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:44.452+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:44.508 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:44.508 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:44.516 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:44.557 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:44.555+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:44.635 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:44.633+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:44.656 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:44.656 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:44.664 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:44.736 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:44.734+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:44.761 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:44.762 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:44.770 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:44.837 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:44.835+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:44.870 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:44.870 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:44.879 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:44.938 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:44.936+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:44.988 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:44.989 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:44.998 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:45.039 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:45.037+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:45.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:44:45.140 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:45.137+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:45.156 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:45.156 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:45.167 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:45.241 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:45.239+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:45.268 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:45.269 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:45.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:45.342 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:45.339+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:45.416 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:45.416 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:45.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:45.442 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:45.440+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:45.522 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:45.522 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:45.530 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:45.575 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:45.573+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:45.627 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:45.627 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:45.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:45.680 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:45.678+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:45.778 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:45.778 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:45.782 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:45.780+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:45.786 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:45.883 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:45.881+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:45.923 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:45.923 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:45.931 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:45.984 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:45.982+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:46.069 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:46.069 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:46.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:44:46.079 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T11:44:46.080 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:46.094 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:46.092+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:46.186 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:46.183+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:46.205 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:46.205 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:46.221 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:46.221 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:46.236 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:46.236 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:46.241 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:46.241 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:46.250 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:46.252 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:46.252 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:46.267 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:46.268 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:46.283 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:46.283 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:46.287 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:46.285+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:46.313 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:46.313 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:46.339 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:46.339 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:46.367 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:46.367 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:46.375 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:46.388 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:46.385+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:46.489 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:46.486+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:46.528 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:46.528 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:46.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:46.528 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:46.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.528 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19545 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4415488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.528 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:46.529 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 51.001490826999998, 2024-09-16T11:44:46.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043429581000000002, 2024-09-16T11:44:46.529 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.529 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.529 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19545 2024-09-16T11:44:46.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.529 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.529 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523839+0000", 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0220000000000004e-06 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523845+0000", 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6500000000000001e-06 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523848+0000", 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1879999999999998e-06 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567092+0000", 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043244145999999997 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567176+0000", 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3825999999999993e-05 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567263+0000", 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7423999999999995e-05 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.532 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19546 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4734976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 51.001472987, 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043599231000000002, 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19546 2024-09-16T11:44:46.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.534 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:46.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:46.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523954+0000", 2024-09-16T11:44:46.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010217099999999999 2024-09-16T11:44:46.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523957+0000", 2024-09-16T11:44:46.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3270000000000001e-06 2024-09-16T11:44:46.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523966+0000", 2024-09-16T11:44:46.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6489999999999997e-06 2024-09-16T11:44:46.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.536 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.536 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523971+0000", 2024-09-16T11:44:46.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:46.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.536 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.536 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567291+0000", 2024-09-16T11:44:46.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043319931999999998 2024-09-16T11:44:46.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.536 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.536 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567360+0000", 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8628999999999996e-05 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567451+0000", 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1012000000000002e-05 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19547 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4841472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:46.537 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 51.001348129999997, 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043658858000000002, 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19547 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525102+0000", 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011257730000000001 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525106+0000", 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9910000000000004e-06 2024-09-16T11:44:46.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525116+0000", 2024-09-16T11:44:46.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0231000000000001e-05 2024-09-16T11:44:46.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525137+0000", 2024-09-16T11:44:46.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0828000000000001e-05 2024-09-16T11:44:46.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567478+0000", 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.042340467999999999 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567554+0000", 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6074000000000004e-05 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567635+0000", 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1493000000000001e-05 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.541 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.542 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.542 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19553 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4947968~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.542 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:46.542 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 51.000121067999999, 2024-09-16T11:44:46.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043450397000000002, 2024-09-16T11:44:46.542 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.542 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.542 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.542 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.542 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.542 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19553 2024-09-16T11:44:46.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.543 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:46.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:46.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:46.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1799999999999999e-07 2024-09-16T11:44:46.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525205+0000", 2024-09-16T11:44:46.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.062e-06 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525208+0000", 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8739999999999999e-06 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525209+0000", 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.725e-06 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568520+0000", 2024-09-16T11:44:46.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043310226 2024-09-16T11:44:46.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568569+0000", 2024-09-16T11:44:46.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8959999999999999e-05 2024-09-16T11:44:46.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568654+0000", 2024-09-16T11:44:46.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5332000000000002e-05 2024-09-16T11:44:46.546 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.546 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19554 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5267456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 51.000112076000001, 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043659587, 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19554 2024-09-16T11:44:46.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.548 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:46.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:46.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527103+0000", 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018907419999999999 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527108+0000", 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6040000000000003e-06 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527118+0000", 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527122+0000", 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9559999999999996e-06 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568688+0000", 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041565116999999999 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568768+0000", 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0474000000000002e-05 2024-09-16T11:44:46.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568872+0000", 2024-09-16T11:44:46.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104028 2024-09-16T11:44:46.551 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.551 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.551 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.551 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19565 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6332416~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.551 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:46.551 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.998033503999999, 2024-09-16T11:44:46.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043494890000000001, 2024-09-16T11:44:46.552 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.552 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.552 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.552 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.552 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.552 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19565 2024-09-16T11:44:46.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.552 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:46.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:46.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527292+0000", 2024-09-16T11:44:46.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.27e-07 2024-09-16T11:44:46.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527293+0000", 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.917e-06 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527299+0000", 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6699999999999999e-06 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527304+0000", 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6160000000000004e-06 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570683+0000", 2024-09-16T11:44:46.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043379171000000001 2024-09-16T11:44:46.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570733+0000", 2024-09-16T11:44:46.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0485000000000002e-05 2024-09-16T11:44:46.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570786+0000", 2024-09-16T11:44:46.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2604e-05 2024-09-16T11:44:46.555 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19566 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6758400~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.998017169999997, 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043643183000000002, 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.556 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19566 2024-09-16T11:44:46.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.557 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:46.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:46.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527308+0000", 2024-09-16T11:44:46.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:44:46.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527310+0000", 2024-09-16T11:44:46.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.68e-06 2024-09-16T11:44:46.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527316+0000", 2024-09-16T11:44:46.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9730000000000002e-06 2024-09-16T11:44:46.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527319+0000", 2024-09-16T11:44:46.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.027e-06 2024-09-16T11:44:46.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570810+0000", 2024-09-16T11:44:46.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043491525000000003 2024-09-16T11:44:46.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570882+0000", 2024-09-16T11:44:46.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:44:46.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.560 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570951+0000", 2024-09-16T11:44:46.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8782000000000007e-05 2024-09-16T11:44:46.560 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.560 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.560 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.560 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19567 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.561 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:46.561 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.998002464000002, 2024-09-16T11:44:46.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043786291999999997, 2024-09-16T11:44:46.561 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.561 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.561 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.561 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.561 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.561 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19567 2024-09-16T11:44:46.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.561 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529108+0000", 2024-09-16T11:44:46.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001785656 2024-09-16T11:44:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529113+0000", 2024-09-16T11:44:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1080000000000001e-06 2024-09-16T11:44:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529122+0000", 2024-09-16T11:44:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0550000000000005e-06 2024-09-16T11:44:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529127+0000", 2024-09-16T11:44:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9849999999999997e-06 2024-09-16T11:44:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570997+0000", 2024-09-16T11:44:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041869811 2024-09-16T11:44:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571063+0000", 2024-09-16T11:44:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6340999999999994e-05 2024-09-16T11:44:46.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571108+0000", 2024-09-16T11:44:46.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5336000000000001e-05 2024-09-16T11:44:46.565 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.565 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.565 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.565 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19580 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 8036352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.565 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:46.565 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.996027163000001, 2024-09-16T11:44:46.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043451596000000002, 2024-09-16T11:44:46.566 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.566 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.566 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.566 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.566 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.566 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19580 2024-09-16T11:44:46.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.566 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:46.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.567 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:46.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.567 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531106+0000", 2024-09-16T11:44:46.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001808179 2024-09-16T11:44:46.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.567 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531110+0000", 2024-09-16T11:44:46.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5990000000000004e-06 2024-09-16T11:44:46.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531123+0000", 2024-09-16T11:44:46.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2687000000000001e-05 2024-09-16T11:44:46.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531129+0000", 2024-09-16T11:44:46.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:44:46.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572642+0000", 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041513474000000002 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572691+0000", 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8989000000000001e-05 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572749+0000", 2024-09-16T11:44:46.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7806999999999997e-05 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19709 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4849664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.993967122000001, 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043537291999999998, 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.570 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19709 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.572 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531358+0000", 2024-09-16T11:44:46.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:46.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.572 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.572 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531360+0000", 2024-09-16T11:44:46.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:44:46.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.572 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.572 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531365+0000", 2024-09-16T11:44:46.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.716e-06 2024-09-16T11:44:46.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531369+0000", 2024-09-16T11:44:46.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5410000000000001e-06 2024-09-16T11:44:46.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574763+0000", 2024-09-16T11:44:46.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394359 2024-09-16T11:44:46.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574831+0000", 2024-09-16T11:44:46.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7997999999999995e-05 2024-09-16T11:44:46.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574895+0000", 2024-09-16T11:44:46.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3427000000000001e-05 2024-09-16T11:44:46.574 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.574 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.574 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.574 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19710 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4956160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.574 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:46.575 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.993951842999998, 2024-09-16T11:44:46.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043674198999999997, 2024-09-16T11:44:46.575 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.575 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.575 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.575 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.575 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.575 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19710 2024-09-16T11:44:46.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.575 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:46.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.576 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:46.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.576 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533105+0000", 2024-09-16T11:44:46.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001731961 2024-09-16T11:44:46.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.577 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.577 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533109+0000", 2024-09-16T11:44:46.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3080000000000001e-06 2024-09-16T11:44:46.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.577 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.577 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533120+0000", 2024-09-16T11:44:46.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0625000000000001e-05 2024-09-16T11:44:46.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.577 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.577 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533125+0000", 2024-09-16T11:44:46.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:44:46.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574923+0000", 2024-09-16T11:44:46.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041797187 2024-09-16T11:44:46.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574991+0000", 2024-09-16T11:44:46.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8752999999999997e-05 2024-09-16T11:44:46.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.575047+0000", 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5596000000000001e-05 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19719 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5488640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.991980093999999, 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043412013999999999, 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.579 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.580 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.580 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.580 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.580 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19719 2024-09-16T11:44:46.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.580 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:46.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.581 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:46.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.581 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.581 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533345+0000", 2024-09-16T11:44:46.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:44:46.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.581 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.581 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533347+0000", 2024-09-16T11:44:46.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8139999999999999e-06 2024-09-16T11:44:46.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533352+0000", 2024-09-16T11:44:46.582 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4970000000000001e-06 2024-09-16T11:44:46.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533355+0000", 2024-09-16T11:44:46.582 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.711e-06 2024-09-16T11:44:46.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576629+0000", 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043274295999999997 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576706+0000", 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6711000000000005e-05 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576757+0000", 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0509e-05 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.583 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.584 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19720 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5808128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.584 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:46.584 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.991967445, 2024-09-16T11:44:46.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043577006000000001, 2024-09-16T11:44:46.584 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.584 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.584 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.584 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.584 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.584 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19720 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:46.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.35e-07 2024-09-16T11:44:46.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.586 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533359+0000", 2024-09-16T11:44:46.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:44:46.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.586 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533364+0000", 2024-09-16T11:44:46.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4349999999999996e-06 2024-09-16T11:44:46.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.587 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533367+0000", 2024-09-16T11:44:46.587 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2349999999999999e-06 2024-09-16T11:44:46.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.587 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576786+0000", 2024-09-16T11:44:46.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043418885999999997 2024-09-16T11:44:46.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576862+0000", 2024-09-16T11:44:46.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6172000000000002e-05 2024-09-16T11:44:46.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576934+0000", 2024-09-16T11:44:46.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1891999999999995e-05 2024-09-16T11:44:46.588 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.588 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.589 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.589 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19721 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6234112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.589 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:46.589 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.991953924999997, 2024-09-16T11:44:46.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043681656999999999, 2024-09-16T11:44:46.589 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.589 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.589 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.589 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.589 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.589 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19721 2024-09-16T11:44:46.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.590 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:46.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:46.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.591 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:46.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-07 2024-09-16T11:44:46.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.591 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533373+0000", 2024-09-16T11:44:46.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.973e-06 2024-09-16T11:44:46.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.591 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533377+0000", 2024-09-16T11:44:46.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.403e-06 2024-09-16T11:44:46.592 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.592 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.592 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533379+0000", 2024-09-16T11:44:46.592 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.058e-06 2024-09-16T11:44:46.592 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.592 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.592 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576959+0000", 2024-09-16T11:44:46.592 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043579433000000001 2024-09-16T11:44:46.592 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.592 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577015+0000", 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5859000000000002e-05 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577052+0000", 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7620999999999998e-05 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19722 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6340608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.593 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:46.594 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.991943198000001, 2024-09-16T11:44:46.594 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043854922999999997, 2024-09-16T11:44:46.594 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.594 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.594 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.594 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.594 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.594 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19722 2024-09-16T11:44:46.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.594 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.594 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.594 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535103+0000", 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017217020000000001 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535107+0000", 2024-09-16T11:44:46.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2819999999999998e-06 2024-09-16T11:44:46.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535118+0000", 2024-09-16T11:44:46.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0148e-05 2024-09-16T11:44:46.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535123+0000", 2024-09-16T11:44:46.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:44:46.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577091+0000", 2024-09-16T11:44:46.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041967930000000001 2024-09-16T11:44:46.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577163+0000", 2024-09-16T11:44:46.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1853999999999993e-05 2024-09-16T11:44:46.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.598 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577236+0000", 2024-09-16T11:44:46.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3699e-05 2024-09-16T11:44:46.598 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.598 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.598 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.598 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19732 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6447104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.598 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:46.598 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.990083341999998, 2024-09-16T11:44:46.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043399048000000003, 2024-09-16T11:44:46.598 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19732 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:46.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535242+0000", 2024-09-16T11:44:46.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:46.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535243+0000", 2024-09-16T11:44:46.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.793e-06 2024-09-16T11:44:46.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535249+0000", 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.643e-06 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535252+0000", 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578501+0000", 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043248343000000002 2024-09-16T11:44:46.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578576+0000", 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5544999999999996e-05 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578640+0000", 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4018999999999998e-05 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19733 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6873088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.990062500999997, 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043549681, 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19733 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535263+0000", 2024-09-16T11:44:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:44:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535265+0000", 2024-09-16T11:44:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9369999999999998e-06 2024-09-16T11:44:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535271+0000", 2024-09-16T11:44:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9719999999999997e-06 2024-09-16T11:44:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535274+0000", 2024-09-16T11:44:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:44:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578668+0000", 2024-09-16T11:44:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043393355000000002 2024-09-16T11:44:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578744+0000", 2024-09-16T11:44:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6853000000000004e-05 2024-09-16T11:44:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578812+0000", 2024-09-16T11:44:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.737e-05 2024-09-16T11:44:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19734 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7299072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.990046753999998, 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043699189999999999, 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19734 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537101+0000", 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001823098 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537105+0000", 2024-09-16T11:44:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537119+0000", 2024-09-16T11:44:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614999999999999e-05 2024-09-16T11:44:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537125+0000", 2024-09-16T11:44:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:44:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578838+0000", 2024-09-16T11:44:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041712605999999999 2024-09-16T11:44:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578912+0000", 2024-09-16T11:44:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4396999999999999e-05 2024-09-16T11:44:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578977+0000", 2024-09-16T11:44:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4858999999999997e-05 2024-09-16T11:44:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19851 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4431872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 50.987966010000001, 2024-09-16T11:44:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043564540999999998, 2024-09-16T11:44:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19851 2024-09-16T11:44:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539100+0000", 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017412079999999999 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539105+0000", 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539116+0000", 2024-09-16T11:44:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1824999999999999e-05 2024-09-16T11:44:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539122+0000", 2024-09-16T11:44:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9859999999999999e-06 2024-09-16T11:44:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580764+0000", 2024-09-16T11:44:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041641299 2024-09-16T11:44:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580839+0000", 2024-09-16T11:44:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5167999999999996e-05 2024-09-16T11:44:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580923+0000", 2024-09-16T11:44:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4314999999999995e-05 2024-09-16T11:44:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55904 4.14 4:2d2dfe38:::rbd_data.11c1b01b799a.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2068993~28159] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T11:44:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.43358911, 2024-09-16T11:44:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.90010917700000004, 2024-09-16T11:44:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55904 2024-09-16T11:44:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091737+0000", 2024-09-16T11:44:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.254e-06 2024-09-16T11:44:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091759+0000", 2024-09-16T11:44:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2487999999999999e-05 2024-09-16T11:44:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091769+0000", 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0199e-05 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091774+0000", 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091830+0000", 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6700999999999998e-05 2024-09-16T11:44:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:44:46.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091840+0000", 2024-09-16T11:44:46.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0128e-05 2024-09-16T11:44:46.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:46.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990355+0000", 2024-09-16T11:44:46.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.89851413099999999 2024-09-16T11:44:46.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990509+0000", 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000154442 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990626+0000", 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000116928 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991526+0000", 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00089961800000000003 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991793+0000", 2024-09-16T11:44:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026779000000000001 2024-09-16T11:44:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991824+0000", 2024-09-16T11:44:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0380999999999999e-05 2024-09-16T11:44:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991845+0000", 2024-09-16T11:44:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0919000000000001e-05 2024-09-16T11:44:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:46.625 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:46.626 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:46.627 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:46.624+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:46.727 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:46.725+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:46.728 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:46.728 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:46.729 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:46.729 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:46.737 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:46.828 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:46.826+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:46.835 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:46.835 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:46.835 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:46.835 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:46.835 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:46.843 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:46.929 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:46.927+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:46.984 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620786172000003, 2024-09-16T11:44:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:44:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:44:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:44:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:44:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:44:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:44:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:44:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:44:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:44:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:44:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:44:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:44:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620764143000002, 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620749377999999, 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:44:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620729310000002, 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:44:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:44:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:44:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:44:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:44:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:44:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:44:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:44:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:44:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620698474000001, 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:44:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:44:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:44:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:44:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:44:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620678953999999, 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:44:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:44:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:44:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:44:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:44:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:44:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:44:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:44:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:44:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:44:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620651375000001, 2024-09-16T11:44:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:44:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:44:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:44:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:44:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:44:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.62063268, 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:44:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:44:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:44:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:44:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:44:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:44:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:44:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:44:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:44:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:44:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:44:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:44:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:44:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:44:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620617670999998, 2024-09-16T11:44:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:44:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:44:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:44:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:44:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:44:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:44:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620585573, 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:44:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:44:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:44:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:44:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:44:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:44:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:44:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:44:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620561954000003, 2024-09-16T11:44:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:44:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:44:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:44:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:44:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620547225000003, 2024-09-16T11:44:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:44:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:44:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:44:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:44:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:44:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:44:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:44:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620525401000002, 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:44:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:44:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:44:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:44:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:44:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:47.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:44:47.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:44:47.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.045 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.045 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.045 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:47.045 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620510750999998, 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:44:47.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:44:47.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620489178, 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:47.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:47.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:47.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:44:47.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:44:47.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:44:47.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:44:47.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:44:47.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620469731999997, 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.054 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:47.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620433034999998, 2024-09-16T11:44:47.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.062 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620411089000001, 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:44:47.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620389203000002, 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:44:47.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:44:47.070 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.620374484000003, 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:44:47.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.072 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:47.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:47.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:44:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:44:47.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:44:47.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:44:47.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:44:47.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.075 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.075 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.075 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:47.077 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:47.077 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:47.074+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:47.169 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:44:47.169 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:44:47.169 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-16T11:44:47.169 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:44:47.169 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:44:47.170 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-16T11:44:47.177 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:47.175+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:47.217 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:47.217 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:47.217 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:47.217 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:47.226 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:47.270 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:47.271 INFO:teuthology.orchestra.run.smithi192.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T11:44:47.271 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:47.278 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:47.276+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:47.326 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:47.326 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:47.326 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:47.326 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:47.326 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:47.334 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:47.379 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:47.377+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:47.433 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19943 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2416640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904911386000002, 2024-09-16T11:44:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027947257999999999, 2024-09-16T11:44:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19943 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5999999999999999e-07 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526297+0000", 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8059999999999999e-06 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526303+0000", 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6579999999999999e-06 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526306+0000", 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3969999999999998e-06 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554165+0000", 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027858519000000002 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554201+0000", 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5908000000000002e-05 2024-09-16T11:44:47.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554242+0000", 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1510000000000001e-05 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19944 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2842624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904896491000002, 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028052639000000001, 2024-09-16T11:44:47.438 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19944 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526312+0000", 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526317+0000", 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5910000000000003e-06 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526321+0000", 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2339999999999999e-06 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554263+0000", 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027942181 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554307+0000", 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4499999999999997e-05 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554362+0000", 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5078999999999999e-05 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19945 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3162112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.442 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904881936000002, 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028145673999999999, 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19945 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:44:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526325+0000", 2024-09-16T11:44:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:44:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526326+0000", 2024-09-16T11:44:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.733e-06 2024-09-16T11:44:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526332+0000", 2024-09-16T11:44:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:44:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526336+0000", 2024-09-16T11:44:47.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8410000000000002e-06 2024-09-16T11:44:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554389+0000", 2024-09-16T11:44:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028053576 2024-09-16T11:44:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554427+0000", 2024-09-16T11:44:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7299999999999999e-05 2024-09-16T11:44:47.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554470+0000", 2024-09-16T11:44:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3195000000000002e-05 2024-09-16T11:44:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19946 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3268608~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:47.447 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904866357000003, 2024-09-16T11:44:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028232587, 2024-09-16T11:44:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19946 2024-09-16T11:44:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:44:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526346+0000", 2024-09-16T11:44:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6459999999999998e-06 2024-09-16T11:44:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526352+0000", 2024-09-16T11:44:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1070000000000001e-06 2024-09-16T11:44:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526355+0000", 2024-09-16T11:44:47.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:44:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554493+0000", 2024-09-16T11:44:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.02813746 2024-09-16T11:44:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554530+0000", 2024-09-16T11:44:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6881999999999997e-05 2024-09-16T11:44:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554572+0000", 2024-09-16T11:44:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2688999999999997e-05 2024-09-16T11:44:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19947 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4014080~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904847003, 2024-09-16T11:44:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028315646999999999, 2024-09-16T11:44:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.452 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19947 2024-09-16T11:44:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:44:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526360+0000", 2024-09-16T11:44:47.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.01e-07 2024-09-16T11:44:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526361+0000", 2024-09-16T11:44:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8300000000000001e-06 2024-09-16T11:44:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526369+0000", 2024-09-16T11:44:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:44:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526377+0000", 2024-09-16T11:44:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0909999999999993e-06 2024-09-16T11:44:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554593+0000", 2024-09-16T11:44:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028216657999999999 2024-09-16T11:44:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554632+0000", 2024-09-16T11:44:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8828000000000002e-05 2024-09-16T11:44:47.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554675+0000", 2024-09-16T11:44:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2710000000000003e-05 2024-09-16T11:44:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19948 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3588096~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904824603999998, 2024-09-16T11:44:47.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028394545, 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19948 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:44:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9200000000000001e-07 2024-09-16T11:44:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526384+0000", 2024-09-16T11:44:47.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526391+0000", 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5150000000000003e-06 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526394+0000", 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4240000000000002e-06 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554695+0000", 2024-09-16T11:44:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028301218999999999 2024-09-16T11:44:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554731+0000", 2024-09-16T11:44:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6298999999999999e-05 2024-09-16T11:44:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554776+0000", 2024-09-16T11:44:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4675999999999997e-05 2024-09-16T11:44:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.461 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19949 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1351680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904808332000002, 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028490081, 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19949 2024-09-16T11:44:47.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:44:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:44:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526400+0000", 2024-09-16T11:44:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7099999999999999e-06 2024-09-16T11:44:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526406+0000", 2024-09-16T11:44:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5140000000000001e-06 2024-09-16T11:44:47.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526409+0000", 2024-09-16T11:44:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5360000000000001e-06 2024-09-16T11:44:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554801+0000", 2024-09-16T11:44:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028392378999999999 2024-09-16T11:44:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554843+0000", 2024-09-16T11:44:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1056000000000001e-05 2024-09-16T11:44:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554888+0000", 2024-09-16T11:44:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466999999999997e-05 2024-09-16T11:44:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.466 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19950 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1777664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904793335999997, 2024-09-16T11:44:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028661844999999998, 2024-09-16T11:44:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19950 2024-09-16T11:44:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:44:47.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526415+0000", 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.776e-06 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526422+0000", 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6930000000000001e-06 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526425+0000", 2024-09-16T11:44:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554923+0000", 2024-09-16T11:44:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028498102000000001 2024-09-16T11:44:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555000+0000", 2024-09-16T11:44:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6619999999999995e-05 2024-09-16T11:44:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555075+0000", 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4931999999999993e-05 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19951 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2097152~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904777525, 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028755718, 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.471 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19951 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:44:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-16T11:44:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526431+0000", 2024-09-16T11:44:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:44:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526441+0000", 2024-09-16T11:44:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7790000000000007e-06 2024-09-16T11:44:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526445+0000", 2024-09-16T11:44:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.634e-06 2024-09-16T11:44:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555100+0000", 2024-09-16T11:44:47.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654722000000001 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555138+0000", 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7996e-05 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555184+0000", 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6427999999999997e-05 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19952 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2203648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904756964999997, 2024-09-16T11:44:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028901157, 2024-09-16T11:44:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19952 2024-09-16T11:44:47.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:44:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526450+0000", 2024-09-16T11:44:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:44:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526452+0000", 2024-09-16T11:44:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9039999999999999e-06 2024-09-16T11:44:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526465+0000", 2024-09-16T11:44:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3559e-05 2024-09-16T11:44:47.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526470+0000", 2024-09-16T11:44:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-09-16T11:44:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555217+0000", 2024-09-16T11:44:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028747287999999999 2024-09-16T11:44:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555272+0000", 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4885000000000001e-05 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555350+0000", 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8428000000000002e-05 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19953 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2523136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.480 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904732373000002, 2024-09-16T11:44:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029004309999999998, 2024-09-16T11:44:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19953 2024-09-16T11:44:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:44:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-16T11:44:47.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526476+0000", 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9810000000000002e-06 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526482+0000", 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526485+0000", 2024-09-16T11:44:47.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2760000000000001e-06 2024-09-16T11:44:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555383+0000", 2024-09-16T11:44:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028898113 2024-09-16T11:44:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555433+0000", 2024-09-16T11:44:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9753999999999998e-05 2024-09-16T11:44:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555478+0000", 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5006999999999998e-05 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19954 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2949120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904717376000001, 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029165054999999999, 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.485 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19954 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1799999999999995e-07 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526491+0000", 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7320000000000001e-06 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526497+0000", 2024-09-16T11:44:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6869999999999999e-06 2024-09-16T11:44:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526500+0000", 2024-09-16T11:44:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:44:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555513+0000", 2024-09-16T11:44:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029012626 2024-09-16T11:44:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555576+0000", 2024-09-16T11:44:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3288999999999997e-05 2024-09-16T11:44:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555654+0000", 2024-09-16T11:44:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7861999999999994e-05 2024-09-16T11:44:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19955 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3375104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904698074000002, 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029241979000000001, 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19955 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:44:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526509+0000", 2024-09-16T11:44:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-09-16T11:44:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526510+0000", 2024-09-16T11:44:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.844e-06 2024-09-16T11:44:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526516+0000", 2024-09-16T11:44:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9039999999999997e-06 2024-09-16T11:44:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526520+0000", 2024-09-16T11:44:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3639999999999999e-06 2024-09-16T11:44:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555669+0000", 2024-09-16T11:44:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029149628 2024-09-16T11:44:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555706+0000", 2024-09-16T11:44:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7017000000000002e-05 2024-09-16T11:44:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555750+0000", 2024-09-16T11:44:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3856999999999997e-05 2024-09-16T11:44:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19956 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3694592~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904683118999998, 2024-09-16T11:44:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029398799999999999, 2024-09-16T11:44:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19956 2024-09-16T11:44:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:44:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526524+0000", 2024-09-16T11:44:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:44:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526525+0000", 2024-09-16T11:44:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-16T11:44:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526531+0000", 2024-09-16T11:44:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6010000000000002e-06 2024-09-16T11:44:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526535+0000", 2024-09-16T11:44:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.597e-06 2024-09-16T11:44:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555785+0000", 2024-09-16T11:44:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029250235999999999 2024-09-16T11:44:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555845+0000", 2024-09-16T11:44:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0393000000000003e-05 2024-09-16T11:44:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555922+0000", 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6710000000000002e-05 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19957 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4120576~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.904667965999998, 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029481157000000001, 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19957 2024-09-16T11:44:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:44:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526539+0000", 2024-09-16T11:44:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526540+0000", 2024-09-16T11:44:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.759e-06 2024-09-16T11:44:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526546+0000", 2024-09-16T11:44:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1079999999999998e-06 2024-09-16T11:44:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526550+0000", 2024-09-16T11:44:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.269e-06 2024-09-16T11:44:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555939+0000", 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029389444000000001 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555977+0000", 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7487999999999998e-05 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.556019+0000", 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2756999999999997e-05 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:28959 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.421811554000001, 2024-09-16T11:44:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031145285000000002, 2024-09-16T11:44:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 28959 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009396+0000", 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.296e-06 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030864+0000", 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.021468451 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030892+0000", 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7555999999999998e-05 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030902+0000", 2024-09-16T11:44:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9550000000000004e-06 2024-09-16T11:44:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030979+0000", 2024-09-16T11:44:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7137000000000004e-05 2024-09-16T11:44:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031051+0000", 2024-09-16T11:44:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1475000000000003e-05 2024-09-16T11:44:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031389+0000", 2024-09-16T11:44:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033827100000000001 2024-09-16T11:44:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.039264+0000", 2024-09-16T11:44:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078751700000000008 2024-09-16T11:44:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040484+0000", 2024-09-16T11:44:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012199579999999999 2024-09-16T11:44:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040517+0000", 2024-09-16T11:44:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2960999999999998e-05 2024-09-16T11:44:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040540+0000", 2024-09-16T11:44:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3054999999999999e-05 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30605 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 3915776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.187505295000001, 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938655999999999, 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30605 2024-09-16T11:44:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:44:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243704+0000", 2024-09-16T11:44:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2199999999999999e-06 2024-09-16T11:44:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243710+0000", 2024-09-16T11:44:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6340000000000003e-06 2024-09-16T11:44:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243714+0000", 2024-09-16T11:44:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5030000000000002e-06 2024-09-16T11:44:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271450+0000", 2024-09-16T11:44:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027736035999999999 2024-09-16T11:44:47.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271536+0000", 2024-09-16T11:44:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6521999999999996e-05 2024-09-16T11:44:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271640+0000", 2024-09-16T11:44:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103341 2024-09-16T11:44:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30607 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4046848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.18748214, 2024-09-16T11:44:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028130446, 2024-09-16T11:44:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30607 2024-09-16T11:44:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.515 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:44:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243725+0000", 2024-09-16T11:44:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2399999999999998e-07 2024-09-16T11:44:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243727+0000", 2024-09-16T11:44:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3379999999999999e-06 2024-09-16T11:44:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243733+0000", 2024-09-16T11:44:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4280000000000001e-06 2024-09-16T11:44:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243737+0000", 2024-09-16T11:44:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8990000000000003e-06 2024-09-16T11:44:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271676+0000", 2024-09-16T11:44:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938481000000001 2024-09-16T11:44:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271757+0000", 2024-09-16T11:44:47.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1396999999999993e-05 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271855+0000", 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7379000000000004e-05 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30608 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.187465289000002, 2024-09-16T11:44:47.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028323879, 2024-09-16T11:44:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30608 2024-09-16T11:44:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243743+0000", 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.762e-06 2024-09-16T11:44:47.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243749+0000", 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5670000000000002e-06 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243758+0000", 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8309999999999996e-06 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271879+0000", 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028120501999999999 2024-09-16T11:44:47.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271965+0000", 2024-09-16T11:44:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5755999999999997e-05 2024-09-16T11:44:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.272065+0000", 2024-09-16T11:44:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000100101 2024-09-16T11:44:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:56045 4.15 4:ab2ce1b7:::rbd_data.11c1b01b799a.0000000000000020:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-16T11:44:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:44:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.252182038000001, 2024-09-16T11:44:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032248165000000002, 2024-09-16T11:44:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 56045 2024-09-16T11:44:47.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:44:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:44:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179026+0000", 2024-09-16T11:44:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.556e-06 2024-09-16T11:44:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179032+0000", 2024-09-16T11:44:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5379999999999999e-06 2024-09-16T11:44:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179045+0000", 2024-09-16T11:44:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.277e-05 2024-09-16T11:44:47.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179067+0000", 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.243e-05 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179120+0000", 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2726000000000003e-05 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179279+0000", 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015865899999999999 2024-09-16T11:44:47.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.185163+0000", 2024-09-16T11:44:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0058837639999999997 2024-09-16T11:44:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211197+0000", 2024-09-16T11:44:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026034130999999999 2024-09-16T11:44:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:47.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211206+0000", 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2539999999999998e-06 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211245+0000", 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8924999999999997e-05 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211272+0000", 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7412000000000001e-05 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.530 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.530 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:47.531 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:47.532 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:47.528+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:47.631 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:47.629+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:47.670 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:47.670 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:47.670 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:47.671 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:47.679 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:47.732 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:47.730+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:47.776 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:47.776 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:47.777 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:47.777 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:47.777 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:47.785 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:47.834 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:47.832+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:47.882 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:47.883 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:47.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:47.883 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:47.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.883 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19282 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.883 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:47.883 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.407232385999997, 2024-09-16T11:44:47.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521151000000002, 2024-09-16T11:44:47.883 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.883 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.883 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.883 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.884 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.884 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19282 2024-09-16T11:44:47.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.884 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:47.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:47.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472838+0000", 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472848+0000", 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9799999999999993e-06 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472858+0000", 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9429999999999995e-06 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472862+0000", 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4460000000000003e-06 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501277+0000", 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028414900999999999 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501322+0000", 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4880999999999999e-05 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501358+0000", 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6245000000000003e-05 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19286 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7979008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.887 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.406859453999999, 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521860999999999, 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19286 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:47.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473211+0000", 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473214+0000", 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1080000000000001e-06 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473226+0000", 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2441e-05 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473234+0000", 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729999999999998e-06 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501592+0000", 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028358491999999999 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501671+0000", 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8846999999999999e-05 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501732+0000", 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1118000000000007e-05 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.891 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.892 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20619 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7610368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.892 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:47.892 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.319510286000003, 2024-09-16T11:44:47.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028500152000000001, 2024-09-16T11:44:47.892 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.892 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.892 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.892 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.892 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.892 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20619 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560560+0000", 2024-09-16T11:44:47.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9499999999999998e-07 2024-09-16T11:44:47.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560561+0000", 2024-09-16T11:44:47.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.071e-06 2024-09-16T11:44:47.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560565+0000", 2024-09-16T11:44:47.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:44:47.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560567+0000", 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588949+0000", 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381878999999999 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588989+0000", 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0649000000000003e-05 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589059+0000", 2024-09-16T11:44:47.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0239000000000003e-05 2024-09-16T11:44:47.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.896 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.896 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20620 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7716864~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:47.896 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:47.896 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.319500886, 2024-09-16T11:44:47.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028627621999999998, 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20620 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:47.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562377+0000", 2024-09-16T11:44:47.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018081729999999999 2024-09-16T11:44:47.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562380+0000", 2024-09-16T11:44:47.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0469999999999998e-06 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562386+0000", 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2040000000000002e-06 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562390+0000", 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589092+0000", 2024-09-16T11:44:47.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026701798999999998 2024-09-16T11:44:47.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589139+0000", 2024-09-16T11:44:47.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7320000000000001e-05 2024-09-16T11:44:47.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589196+0000", 2024-09-16T11:44:47.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7201999999999999e-05 2024-09-16T11:44:47.900 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.900 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.900 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41954 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465795096000001, 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028594441000000002, 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41954 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.901 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414275+0000", 2024-09-16T11:44:47.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414277+0000", 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414282+0000", 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7559999999999996e-06 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414286+0000", 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442778+0000", 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028492237 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442816+0000", 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8198000000000003e-05 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442869+0000", 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2589000000000001e-05 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41955 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465780037999998, 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028687865, 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41955 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414292+0000", 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414299+0000", 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4279999999999999e-06 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414303+0000", 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442892+0000", 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028589564000000001 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442930+0000", 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330999999999999e-05 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442977+0000", 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7874000000000003e-05 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41956 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:47.909 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465744721, 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028766896, 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41956 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:47.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414326+0000", 2024-09-16T11:44:47.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9900000000000009e-07 2024-09-16T11:44:47.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414330+0000", 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2339999999999996e-06 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414341+0000", 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1358999999999999e-05 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414347+0000", 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8100000000000003e-06 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443002+0000", 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654331000000002 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443040+0000", 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8262000000000001e-05 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443092+0000", 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1901000000000003e-05 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.913 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.914 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41957 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.914 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:47.914 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465717302000002, 2024-09-16T11:44:47.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028848318000000001, 2024-09-16T11:44:47.914 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.914 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.914 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41957 2024-09-16T11:44:47.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.915 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:47.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:47.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414353+0000", 2024-09-16T11:44:47.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:47.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414355+0000", 2024-09-16T11:44:47.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3489999999999999e-06 2024-09-16T11:44:47.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414364+0000", 2024-09-16T11:44:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1449999999999997e-06 2024-09-16T11:44:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414368+0000", 2024-09-16T11:44:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8940000000000003e-06 2024-09-16T11:44:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443115+0000", 2024-09-16T11:44:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028746532000000002 2024-09-16T11:44:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443152+0000", 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6814999999999999e-05 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443201+0000", 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9069000000000001e-05 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41958 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.918 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465693520000002, 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028939856, 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41958 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:47.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:47.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414377+0000", 2024-09-16T11:44:47.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4099999999999999e-07 2024-09-16T11:44:47.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414379+0000", 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.418e-06 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414385+0000", 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999999e-06 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414389+0000", 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7740000000000002e-06 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443223+0000", 2024-09-16T11:44:47.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028833919 2024-09-16T11:44:47.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443263+0000", 2024-09-16T11:44:47.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0148999999999998e-05 2024-09-16T11:44:47.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443316+0000", 2024-09-16T11:44:47.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2636000000000002e-05 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41959 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465676594999998, 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029040309, 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.924 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41959 2024-09-16T11:44:47.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.924 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:47.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:47.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:47.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:44:47.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414395+0000", 2024-09-16T11:44:47.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7579999999999999e-06 2024-09-16T11:44:47.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414401+0000", 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5330000000000002e-06 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414404+0000", 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.529e-06 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443339+0000", 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028934933999999999 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443383+0000", 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3832000000000003e-05 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443433+0000", 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0309000000000002e-05 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41960 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.927 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465661761, 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029138134, 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41960 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:47.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-07 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414410+0000", 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414417+0000", 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414420+0000", 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443457+0000", 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029037065000000001 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443495+0000", 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7555000000000002e-05 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443546+0000", 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1422000000000003e-05 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.931 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.932 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.932 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41961 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.932 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:47.932 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465645780000003, 2024-09-16T11:44:47.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029230973, 2024-09-16T11:44:47.932 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.932 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.932 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.932 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.932 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.932 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41961 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4499999999999998e-07 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414432+0000", 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.413e-06 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414439+0000", 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4769999999999997e-06 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414443+0000", 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8859999999999997e-06 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443569+0000", 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029126124 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443606+0000", 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6588999999999998e-05 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443655+0000", 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9138999999999998e-05 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41962 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465622813000003, 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029324505000000001, 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.936 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41962 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:47.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414449+0000", 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0279999999999999e-06 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414456+0000", 2024-09-16T11:44:47.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3339999999999998e-06 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414459+0000", 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.382e-06 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443678+0000", 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029218958999999999 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443719+0000", 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0722000000000001e-05 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443771+0000", 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2661000000000003e-05 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41963 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:47.940 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465606909999998, 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029416557999999999, 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41963 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414465+0000", 2024-09-16T11:44:47.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414470+0000", 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4750000000000001e-06 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414474+0000", 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3720000000000001e-06 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443795+0000", 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029320821 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443830+0000", 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5463000000000002e-05 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443879+0000", 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9147000000000002e-05 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.944 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41964 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465592022000003, 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029521202999999999, 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41964 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:44:47.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414480+0000", 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414486+0000", 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414489+0000", 2024-09-16T11:44:47.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2770000000000001e-06 2024-09-16T11:44:47.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443903+0000", 2024-09-16T11:44:47.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029414109000000001 2024-09-16T11:44:47.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443944+0000", 2024-09-16T11:44:47.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0992999999999999e-05 2024-09-16T11:44:47.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443999+0000", 2024-09-16T11:44:47.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4632000000000001e-05 2024-09-16T11:44:47.949 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.949 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.949 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.950 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41965 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.950 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:47.950 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465577060000001, 2024-09-16T11:44:47.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029611293, 2024-09-16T11:44:47.950 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.950 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.950 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.950 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.950 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.951 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41965 2024-09-16T11:44:47.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.951 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:47.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:47.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414495+0000", 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.88e-06 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414501+0000", 2024-09-16T11:44:47.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2489999999999998e-06 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414504+0000", 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3139999999999999e-06 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444023+0000", 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029518564000000001 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444058+0000", 2024-09-16T11:44:47.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5196e-05 2024-09-16T11:44:47.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444104+0000", 2024-09-16T11:44:47.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5732999999999997e-05 2024-09-16T11:44:47.954 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.954 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.954 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.954 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41966 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.954 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465560854000003, 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029707678000000001, 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41966 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:47.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:47.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:47.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:47.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414514+0000", 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414518+0000", 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3839999999999999e-06 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414521+0000", 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0400000000000001e-06 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444128+0000", 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029606612000000001 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444167+0000", 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9418000000000001e-05 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444216+0000", 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9354999999999998e-05 2024-09-16T11:44:47.958 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41967 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465545044999999, 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029796916, 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.959 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41967 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6699999999999999e-07 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414527+0000", 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7060000000000001e-06 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414533+0000", 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2140000000000001e-06 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414536+0000", 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444239+0000", 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029702698999999999 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444273+0000", 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4229e-05 2024-09-16T11:44:47.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444321+0000", 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8477999999999999e-05 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41968 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465530147999999, 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029902765000000001, 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.963 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.964 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41968 2024-09-16T11:44:47.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.964 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:47.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414540+0000", 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414542+0000", 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414547+0000", 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4639999999999997e-06 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414551+0000", 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6279999999999998e-06 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444346+0000", 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029794885 2024-09-16T11:44:47.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444389+0000", 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3442000000000002e-05 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444442+0000", 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3173000000000001e-05 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41969 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.465515236000002, 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029997612999999999, 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41969 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414555+0000", 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414556+0000", 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.725e-06 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414563+0000", 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2530000000000001e-06 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414566+0000", 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4429999999999999e-06 2024-09-16T11:44:47.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444463+0000", 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029897271 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444503+0000", 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9782e-05 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444552+0000", 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8789000000000003e-05 2024-09-16T11:44:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.972 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:47.972 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:47.972 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:47.973 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:47.974 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:47.971+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:48.074 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:48.072+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:48.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:44:48.112 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:48.112 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:48.113 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:48.113 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:48.121 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:48.175 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:48.173+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:48.259 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:48.260 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:48.268 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:48.276 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:48.274+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:48.351 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:44:48.367 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:48.367 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:48.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:48.368 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:48.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.368 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:48.368 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:48.368 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.444729531999997, 2024-09-16T11:44:48.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:44:48.368 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.368 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:48.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:48.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:48.368 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:44:48.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.369 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:48.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:48.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:44:48.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:44:48.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:44:48.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:44:48.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:44:48.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:48.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:44:48.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:44:48.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:44:48.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:44:48.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:44:48.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:44:48.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.373 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.444683839999996, 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:44:48.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:44:48.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:44:48.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:44:48.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:44:48.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:44:48.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:44:48.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:44:48.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:44:48.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:44:48.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:44:48.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:44:48.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:44:48.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:48.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:44:48.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:44:48.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:48.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:44:48.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:44:48.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.379 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.380 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:48.380 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:48.380 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.444626909999997, 2024-09-16T11:44:48.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:44:48.380 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.380 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:48.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:48.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:48.380 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:44:48.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:48.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:44:48.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:44:48.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:44:48.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:44:48.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:44:48.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:44:48.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:44:48.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:44:48.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:44:48.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:44:48.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:48.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:44:48.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:44:48.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:48.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:44:48.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:44:48.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:48.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:44:48.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:44:48.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:44:48.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:44:48.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:44:48.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:44:48.386 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.386 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.386 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.387 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:48.387 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:48.387 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.444601542000001, 2024-09-16T11:44:48.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:44:48.387 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.387 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:48.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:48.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:48.387 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:44:48.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.387 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:44:48.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:44:48.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:48.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:48.392 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.444549574000007, 2024-09-16T11:44:48.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:44:48.393 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.393 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.393 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:48.393 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:48.393 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:48.393 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:44:48.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.393 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:48.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:48.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:48.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:44:48.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:44:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:44:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:44:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:44:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:44:48.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:44:48.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:48.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:44:48.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:44:48.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:48.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:44:48.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:44:48.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.398 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.444468463999996, 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:44:48.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:44:48.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:44:48.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:44:48.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:44:48.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:44:48.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:44:48.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:44:48.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:44:48.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:44:48.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:44:48.404 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.404 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.404 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.404 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.444442486, 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:44:48.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:44:48.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:44:48.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:44:48.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:48.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:44:48.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:44:48.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:44:48.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:44:48.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:44:48.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.410 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.410 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:48.410 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.026923861, 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:48.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:48.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:44:48.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:44:48.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:44:48.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:44:48.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:44:48.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:44:48.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:44:48.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:48.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:48.416 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:48.417 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.026901929999994, 2024-09-16T11:44:48.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:44:48.417 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.417 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:48.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:48.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:48.417 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:44:48.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.417 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:48.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:48.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:44:48.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:44:48.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:44:48.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:44:48.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:44:48.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:44:48.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:44:48.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:48.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:44:48.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:44:48.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:44:48.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:44:48.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:48.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:44:48.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:44:48.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:48.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:44:48.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:44:48.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:48.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:44:48.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:44:48.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.423 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:48.423 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:48.423 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.026825041999999, 2024-09-16T11:44:48.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:44:48.423 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.423 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:48.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:48.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:48.423 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:44:48.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.423 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:44:48.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:44:48.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:44:48.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:44:48.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:44:48.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:48.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:44:48.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:44:48.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:48.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:44:48.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:44:48.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:48.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:44:48.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:44:48.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28187 4.17 e52/28 4:ee916bd9:::rbd_data.11c1ce740b46.0000000000000000:head v 52'195, mlcod=52'193)", 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.855405656000002, 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075854429999999999, 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:48.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509349+0000", 2024-09-16T11:44:48.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2300000000000001e-06 2024-09-16T11:44:48.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513315+0000", 2024-09-16T11:44:48.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039660590000000001 2024-09-16T11:44:48.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513328+0000", 2024-09-16T11:44:48.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.308e-05 2024-09-16T11:44:48.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513334+0000", 2024-09-16T11:44:48.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:48.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513385+0000", 2024-09-16T11:44:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1347000000000002e-05 2024-09-16T11:44:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513407+0000", 2024-09-16T11:44:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1852999999999999e-05 2024-09-16T11:44:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516882+0000", 2024-09-16T11:44:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034749690000000001 2024-09-16T11:44:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516933+0000", 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0952999999999998e-05 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28959 4.8 e53/28 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head v 53'408, mlcod=53'406)", 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.333256599999999, 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088740420000000004, 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.433 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031498+0000", 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.652e-06 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033515+0000", 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020165930000000001 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033539+0000", 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4295999999999999e-05 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033548+0000", 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1239999999999994e-06 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033610+0000", 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1371999999999994e-05 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033637+0000", 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7098999999999999e-05 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040341+0000", 2024-09-16T11:44:48.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067045819999999997 2024-09-16T11:44:48.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040371+0000", 2024-09-16T11:44:48.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9323999999999999e-05 2024-09-16T11:44:48.437 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.437 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.437 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32303 4.8 e53/28 4:1436ac82:::rbd_data.11c1cab081b3.0000000000000022:head v 53'417, mlcod=53'415)", 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.862830625999997, 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085565160000000001, 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501930+0000", 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7649999999999996e-06 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501946+0000", 2024-09-16T11:44:48.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5301000000000001e-05 2024-09-16T11:44:48.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501952+0000", 2024-09-16T11:44:48.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:44:48.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502100+0000", 2024-09-16T11:44:48.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014847600000000001 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502120+0000", 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9303000000000001e-05 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510436+0000", 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083166420000000008 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510479+0000", 2024-09-16T11:44:48.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2979000000000003e-05 2024-09-16T11:44:48.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.442 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.442 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32306 4.1e e53/28 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head v 53'124, mlcod=53'122)", 2024-09-16T11:44:48.442 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:48.442 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.862764632000001, 2024-09-16T11:44:48.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084990900000000008, 2024-09-16T11:44:48.442 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.442 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.442 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:48.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501994+0000", 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502002+0000", 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3839999999999997e-06 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502007+0000", 2024-09-16T11:44:48.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6290000000000001e-06 2024-09-16T11:44:48.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502454+0000", 2024-09-16T11:44:48.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00044736800000000002 2024-09-16T11:44:48.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502471+0000", 2024-09-16T11:44:48.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6648999999999999e-05 2024-09-16T11:44:48.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510455+0000", 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0079841909999999999 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510488+0000", 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2948999999999999e-05 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.446 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32307 4.8 e53/28 4:11c1c37d:::rbd_data.11c1cab081b3.0000000000000026:head v 53'418, mlcod=53'415)", 2024-09-16T11:44:48.447 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:48.447 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.862009297, 2024-09-16T11:44:48.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00776212, 2024-09-16T11:44:48.447 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.447 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.447 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:48.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:48.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:48.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:48.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502750+0000", 2024-09-16T11:44:48.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:44:48.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502760+0000", 2024-09-16T11:44:48.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:44:48.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502767+0000", 2024-09-16T11:44:48.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4219999999999997e-06 2024-09-16T11:44:48.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502802+0000", 2024-09-16T11:44:48.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5089999999999998e-05 2024-09-16T11:44:48.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502816+0000", 2024-09-16T11:44:48.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4092e-05 2024-09-16T11:44:48.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510483+0000", 2024-09-16T11:44:48.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00766704 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510506+0000", 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3195999999999999e-05 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33583 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'416, mlcod=53'411)", 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.154050888, 2024-09-16T11:44:48.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078132459999999994, 2024-09-16T11:44:48.452 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.452 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.452 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:48.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:48.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210703+0000", 2024-09-16T11:44:48.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:48.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210706+0000", 2024-09-16T11:44:48.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-06 2024-09-16T11:44:48.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210712+0000", 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210715+0000", 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210736+0000", 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0293999999999999e-05 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210746+0000", 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0178e-05 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218477+0000", 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007731501 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218516+0000", 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8148000000000001e-05 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.455 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.456 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.456 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33590 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'419, mlcod=53'415)", 2024-09-16T11:44:48.456 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:48.456 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.142992747000001, 2024-09-16T11:44:48.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082499879999999998, 2024-09-16T11:44:48.456 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.456 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.456 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:48.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:48.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221761+0000", 2024-09-16T11:44:48.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-16T11:44:48.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221765+0000", 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1590000000000003e-06 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221774+0000", 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7490000000000004e-06 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221778+0000", 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0419999999999996e-06 2024-09-16T11:44:48.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221800+0000", 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1926999999999999e-05 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221810+0000", 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0079e-05 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.229980+0000", 2024-09-16T11:44:48.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081698469999999992 2024-09-16T11:44:48.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230010+0000", 2024-09-16T11:44:48.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0746e-05 2024-09-16T11:44:48.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.460 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.460 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33591 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'420, mlcod=53'415)", 2024-09-16T11:44:48.460 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:48.460 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.142431262999999, 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007702576, 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222326+0000", 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222337+0000", 2024-09-16T11:44:48.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1008e-05 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222342+0000", 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0050000000000004e-06 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222366+0000", 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3696000000000001e-05 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222385+0000", 2024-09-16T11:44:48.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-16T11:44:48.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230012+0000", 2024-09-16T11:44:48.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076270940000000001 2024-09-16T11:44:48.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230025+0000", 2024-09-16T11:44:48.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2112e-05 2024-09-16T11:44:48.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.464 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:55282 4.1e e73/28 4:785299e2:::rbd_data.11c11a49c2da.0000000000000001:head v 73'324, mlcod=73'322)", 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.125734743999999, 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071431890000000003, 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:44:48.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239020+0000", 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.049e-06 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244238+0000", 2024-09-16T11:44:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052183489999999997 2024-09-16T11:44:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244252+0000", 2024-09-16T11:44:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4246999999999999e-05 2024-09-16T11:44:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244275+0000", 2024-09-16T11:44:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3093999999999999e-05 2024-09-16T11:44:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244314+0000", 2024-09-16T11:44:48.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8495000000000003e-05 2024-09-16T11:44:48.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244330+0000", 2024-09-16T11:44:48.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5917e-05 2024-09-16T11:44:48.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246121+0000", 2024-09-16T11:44:48.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017909239999999999 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246162+0000", 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1114e-05 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:61651 4.11 e83/28 4:8f4a51c5:::rbd_data.11c175f8cab7.0000000000000014:head v 83'694, mlcod=81'692)", 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.751315197, 2024-09-16T11:44:48.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.009466482, 2024-09-16T11:44:48.470 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:48.470 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:48.470 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:48.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:48.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:44:48.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:48.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:44:48.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:48.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:48.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613439+0000", 2024-09-16T11:44:48.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8400000000000003e-07 2024-09-16T11:44:48.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:48.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618867+0000", 2024-09-16T11:44:48.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054276059999999998 2024-09-16T11:44:48.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:48.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618883+0000", 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6265000000000001e-05 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618890+0000", 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2980000000000003e-06 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618950+0000", 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0025999999999998e-05 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:48.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618971+0000", 2024-09-16T11:44:48.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0458999999999998e-05 2024-09-16T11:44:48.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:48.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622869+0000", 2024-09-16T11:44:48.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038983450000000001 2024-09-16T11:44:48.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:48.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:48.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:48.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622905+0000", 2024-09-16T11:44:48.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599000000000003e-05 2024-09-16T11:44:48.473 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.474 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.474 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.474 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:48.474 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:48.474 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:48.476 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T11:44:48.476 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:48.476 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:48.477 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:48.477 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:48.473+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:48.490 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:48.490 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:48.508 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:48.508 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:48.528 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:48.529 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:48.545 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:48.545 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:48.561 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:48.562 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:48.576 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:48.573+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:48.577 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:48.577 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:48.593 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:48.593 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:48.595 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:48.595 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:48.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:48.677 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:48.675+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:48.742 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:48.743 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:48.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:48.794 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:48.792+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:48.849 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:48.849 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:48.857 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:48.879 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:48.877+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:48.980 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:48.978+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:48.995 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:48.996 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:49.004 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:49.081 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:49.079+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:49.101 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:49.101 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:49.109 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:49.182 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:49.180+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:49.207 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:49.207 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:49.216 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:49.283 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:49.281+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:49.314 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:49.315 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:49.323 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:49.384 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:49.382+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:49.464 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:49.464 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:49.474 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:49.485 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:49.483+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:49.578 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:49.578 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:49.586 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:49.583+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:49.587 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:49.684 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:49.685 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:49.687 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:49.685+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:49.693 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:49.788 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:49.786+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:49.830 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:49.831 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:49.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:49.923 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:49.921+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:49.977 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:49.977 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:49.985 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:49.990 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:49.987+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:50.087 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:50.088 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:50.090 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:50.087+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:50.096 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:50.190 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:50.188+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:50.234 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:50.234 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:50.242 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:50.292 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:50.289+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:50.340 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:50.341 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:50.350 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:50.392 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:50.390+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:50.477 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:44:50.495 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:50.495 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:50.505 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:50.521 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:50.519+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:50.595 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:50.593+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:50.605 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:44:50.608 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:50.608 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:50.617 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:50.696 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:50.694+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:50.720 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:50.736 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:50.736 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:50.750 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:50.751 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:50.758 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:50.758 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:50.766 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:50.767 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:50.767 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:50.782 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:50.796 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:50.794+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:50.798 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:50.798 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:50.813 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:50.814 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:50.829 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:50.829 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:50.866 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:50.867 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:50.875 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:50.918 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:50.916+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:50.973 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:50.973 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:50.973 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:50.981 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:51.022 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:51.020+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:51.099 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:51.097+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:51.122 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:51.122 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:51.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:51.122 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:51.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.122 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19282 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:51.122 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:51.122 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.646429108, 2024-09-16T11:44:51.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521151000000002, 2024-09-16T11:44:51.122 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.122 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19282 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:44:51.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472838+0000", 2024-09-16T11:44:51.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:44:51.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472848+0000", 2024-09-16T11:44:51.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9799999999999993e-06 2024-09-16T11:44:51.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472858+0000", 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9429999999999995e-06 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472862+0000", 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4460000000000003e-06 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501277+0000", 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028414900999999999 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501322+0000", 2024-09-16T11:44:51.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4880999999999999e-05 2024-09-16T11:44:51.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501358+0000", 2024-09-16T11:44:51.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6245000000000003e-05 2024-09-16T11:44:51.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.126 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19286 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7979008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.646056176000002, 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521860999999999, 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19286 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:51.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:44:51.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473211+0000", 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473214+0000", 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1080000000000001e-06 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473226+0000", 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2441e-05 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473234+0000", 2024-09-16T11:44:51.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729999999999998e-06 2024-09-16T11:44:51.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501592+0000", 2024-09-16T11:44:51.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028358491999999999 2024-09-16T11:44:51.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501671+0000", 2024-09-16T11:44:51.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8846999999999999e-05 2024-09-16T11:44:51.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501732+0000", 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1118000000000007e-05 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20619 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7610368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.558707007999999, 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028500152000000001, 2024-09-16T11:44:51.131 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.132 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.132 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20619 2024-09-16T11:44:51.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.132 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:51.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560560+0000", 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9499999999999998e-07 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560561+0000", 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.071e-06 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560565+0000", 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560567+0000", 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588949+0000", 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381878999999999 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588989+0000", 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0649000000000003e-05 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589059+0000", 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0239000000000003e-05 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.135 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.136 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20620 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7716864~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:51.136 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:51.136 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.558697608000003, 2024-09-16T11:44:51.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028627621999999998, 2024-09-16T11:44:51.136 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.136 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20620 2024-09-16T11:44:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:44:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562377+0000", 2024-09-16T11:44:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018081729999999999 2024-09-16T11:44:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562380+0000", 2024-09-16T11:44:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0469999999999998e-06 2024-09-16T11:44:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562386+0000", 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2040000000000002e-06 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562390+0000", 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589092+0000", 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026701798999999998 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589139+0000", 2024-09-16T11:44:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7320000000000001e-05 2024-09-16T11:44:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589196+0000", 2024-09-16T11:44:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7201999999999999e-05 2024-09-16T11:44:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.140 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41954 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704991818000003, 2024-09-16T11:44:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028594441000000002, 2024-09-16T11:44:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41954 2024-09-16T11:44:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:44:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414275+0000", 2024-09-16T11:44:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:44:51.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414277+0000", 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414282+0000", 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7559999999999996e-06 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414286+0000", 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:44:51.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442778+0000", 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028492237 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442816+0000", 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8198000000000003e-05 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442869+0000", 2024-09-16T11:44:51.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2589000000000001e-05 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41955 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704976760000001, 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028687865, 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41955 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414292+0000", 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414299+0000", 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4279999999999999e-06 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414303+0000", 2024-09-16T11:44:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:44:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442892+0000", 2024-09-16T11:44:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028589564000000001 2024-09-16T11:44:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442930+0000", 2024-09-16T11:44:51.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330999999999999e-05 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442977+0000", 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7874000000000003e-05 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41956 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704941443000003, 2024-09-16T11:44:51.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028766896, 2024-09-16T11:44:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41956 2024-09-16T11:44:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414326+0000", 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9900000000000009e-07 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414330+0000", 2024-09-16T11:44:51.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2339999999999996e-06 2024-09-16T11:44:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414341+0000", 2024-09-16T11:44:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1358999999999999e-05 2024-09-16T11:44:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414347+0000", 2024-09-16T11:44:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8100000000000003e-06 2024-09-16T11:44:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443002+0000", 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654331000000002 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443040+0000", 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8262000000000001e-05 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443092+0000", 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1901000000000003e-05 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41957 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704914023999997, 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028848318000000001, 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41957 2024-09-16T11:44:51.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414353+0000", 2024-09-16T11:44:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414355+0000", 2024-09-16T11:44:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3489999999999999e-06 2024-09-16T11:44:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414364+0000", 2024-09-16T11:44:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1449999999999997e-06 2024-09-16T11:44:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414368+0000", 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8940000000000003e-06 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443115+0000", 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028746532000000002 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443152+0000", 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6814999999999999e-05 2024-09-16T11:44:51.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443201+0000", 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9069000000000001e-05 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41958 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704890241999998, 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028939856, 2024-09-16T11:44:51.158 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41958 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:44:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414377+0000", 2024-09-16T11:44:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4099999999999999e-07 2024-09-16T11:44:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414379+0000", 2024-09-16T11:44:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.418e-06 2024-09-16T11:44:51.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414385+0000", 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999999e-06 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414389+0000", 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7740000000000002e-06 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443223+0000", 2024-09-16T11:44:51.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028833919 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443263+0000", 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0148999999999998e-05 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443316+0000", 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2636000000000002e-05 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41959 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704873317000001, 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029040309, 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41959 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:44:51.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414395+0000", 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7579999999999999e-06 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414401+0000", 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5330000000000002e-06 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414404+0000", 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.529e-06 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443339+0000", 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028934933999999999 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443383+0000", 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3832000000000003e-05 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443433+0000", 2024-09-16T11:44:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0309000000000002e-05 2024-09-16T11:44:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41960 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704858483000002, 2024-09-16T11:44:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029138134, 2024-09-16T11:44:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.167 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41960 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-07 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414410+0000", 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414417+0000", 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414420+0000", 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443457+0000", 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029037065000000001 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443495+0000", 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7555000000000002e-05 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443546+0000", 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1422000000000003e-05 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.171 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41961 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704842501999998, 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029230973, 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41961 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:51.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:44:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4499999999999998e-07 2024-09-16T11:44:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414432+0000", 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.413e-06 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414439+0000", 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4769999999999997e-06 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414443+0000", 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8859999999999997e-06 2024-09-16T11:44:51.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443569+0000", 2024-09-16T11:44:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029126124 2024-09-16T11:44:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443606+0000", 2024-09-16T11:44:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6588999999999998e-05 2024-09-16T11:44:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443655+0000", 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9138999999999998e-05 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41962 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704819534999999, 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029324505000000001, 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41962 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:44:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:44:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414449+0000", 2024-09-16T11:44:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0279999999999999e-06 2024-09-16T11:44:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414456+0000", 2024-09-16T11:44:51.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3339999999999998e-06 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414459+0000", 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.382e-06 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443678+0000", 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029218958999999999 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443719+0000", 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0722000000000001e-05 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443771+0000", 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2661000000000003e-05 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41963 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:51.180 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704803632000001, 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029416557999999999, 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41963 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:44:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-16T11:44:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414465+0000", 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414470+0000", 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4750000000000001e-06 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414474+0000", 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3720000000000001e-06 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443795+0000", 2024-09-16T11:44:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029320821 2024-09-16T11:44:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443830+0000", 2024-09-16T11:44:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5463000000000002e-05 2024-09-16T11:44:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443879+0000", 2024-09-16T11:44:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9147000000000002e-05 2024-09-16T11:44:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41964 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704788743999998, 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029521202999999999, 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.185 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41964 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:44:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:44:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414480+0000", 2024-09-16T11:44:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:44:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414486+0000", 2024-09-16T11:44:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:51.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414489+0000", 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2770000000000001e-06 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443903+0000", 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029414109000000001 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443944+0000", 2024-09-16T11:44:51.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0992999999999999e-05 2024-09-16T11:44:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443999+0000", 2024-09-16T11:44:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4632000000000001e-05 2024-09-16T11:44:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41965 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:51.189 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704773781999997, 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029611293, 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41965 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414495+0000", 2024-09-16T11:44:51.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.88e-06 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414501+0000", 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2489999999999998e-06 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414504+0000", 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3139999999999999e-06 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444023+0000", 2024-09-16T11:44:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029518564000000001 2024-09-16T11:44:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444058+0000", 2024-09-16T11:44:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5196e-05 2024-09-16T11:44:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444104+0000", 2024-09-16T11:44:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5732999999999997e-05 2024-09-16T11:44:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.193 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41966 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704757575999999, 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029707678000000001, 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41966 2024-09-16T11:44:51.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414514+0000", 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414518+0000", 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3839999999999999e-06 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414521+0000", 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0400000000000001e-06 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444128+0000", 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029606612000000001 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444167+0000", 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9418000000000001e-05 2024-09-16T11:44:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444216+0000", 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9354999999999998e-05 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41967 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704741767000002, 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029796916, 2024-09-16T11:44:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41967 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6699999999999999e-07 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414527+0000", 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7060000000000001e-06 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414533+0000", 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2140000000000001e-06 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414536+0000", 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444239+0000", 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029702698999999999 2024-09-16T11:44:51.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444273+0000", 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4229e-05 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444321+0000", 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8477999999999999e-05 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41968 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704726870000002, 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029902765000000001, 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41968 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414540+0000", 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414542+0000", 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414547+0000", 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4639999999999997e-06 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414551+0000", 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6279999999999998e-06 2024-09-16T11:44:51.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444346+0000", 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029794885 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444389+0000", 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3442000000000002e-05 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444442+0000", 2024-09-16T11:44:51.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3173000000000001e-05 2024-09-16T11:44:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41969 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:44:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.704711957999997, 2024-09-16T11:44:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029997612999999999, 2024-09-16T11:44:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41969 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414555+0000", 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414556+0000", 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.725e-06 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414563+0000", 2024-09-16T11:44:51.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2530000000000001e-06 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414566+0000", 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4429999999999999e-06 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444463+0000", 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029897271 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444503+0000", 2024-09-16T11:44:51.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9782e-05 2024-09-16T11:44:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444552+0000", 2024-09-16T11:44:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8789000000000003e-05 2024-09-16T11:44:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.211 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.211 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:51.213 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:51.213 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:51.211+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:51.310 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:51.311 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:51.313 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:51.311+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:51.318 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:51.414 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:51.411+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:51.417 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:51.418 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:51.418 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:51.418 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:51.418 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:51.426 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:51.449 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T11:44:51.515 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:51.512+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:51.568 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:51.568 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:51.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:51.568 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:51.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.568 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:51.568 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:51.569 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.645372550999994, 2024-09-16T11:44:51.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:44:51.569 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.569 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:51.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:51.569 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:44:51.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.569 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:51.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:51.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:44:51.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:44:51.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:44:51.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:44:51.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:44:51.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:51.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:44:51.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:44:51.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:44:51.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:44:51.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:44:51.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:44:51.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:44:51.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:44:51.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:44:51.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:44:51.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.574 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.575 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:51.575 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:51.575 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.645326858999994, 2024-09-16T11:44:51.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:44:51.575 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.575 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:51.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:51.575 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:44:51.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.575 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:51.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:44:51.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:44:51.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:44:51.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:44:51.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:44:51.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:44:51.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:44:51.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:44:51.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:44:51.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:44:51.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:44:51.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:44:51.580 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.580 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.580 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.581 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:51.581 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:51.581 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.645269928999994, 2024-09-16T11:44:51.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:44:51.581 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.581 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:51.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:51.581 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:44:51.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.582 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:51.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:44:51.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:44:51.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:44:51.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:44:51.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:44:51.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:44:51.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:44:51.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:51.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:44:51.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:44:51.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:51.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:44:51.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:44:51.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.586 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.587 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:51.587 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:51.587 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.645244560999998, 2024-09-16T11:44:51.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:44:51.587 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.587 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:51.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:51.587 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:44:51.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.588 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:51.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:51.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:44:51.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:44:51.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:44:51.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:44:51.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:44:51.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:44:51.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:44:51.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:51.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:44:51.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:44:51.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:51.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:44:51.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:44:51.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:51.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:44:51.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:44:51.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:44:51.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:44:51.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:44:51.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:44:51.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.593 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.593 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:51.593 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:51.593 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.645192593000004, 2024-09-16T11:44:51.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:44:51.593 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.593 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:44:51.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:44:51.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:44:51.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:44:51.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:44:51.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:44:51.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:51.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:44:51.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:44:51.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:44:51.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:44:51.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:44:51.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.645111482999994, 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:51.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:51.600 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:44:51.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.600 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:51.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:51.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:44:51.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:44:51.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:44:51.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:44:51.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:44:51.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:44:51.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:51.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:44:51.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:44:51.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:51.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:44:51.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:44:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:44:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:44:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:44:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:44:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:44:51.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:44:51.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.605 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.605 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:51.605 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:51.605 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.645085504999997, 2024-09-16T11:44:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:44:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:44:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:51.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:44:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:44:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:44:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:44:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:44:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:44:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:44:51.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:44:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:44:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:44:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:44:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:51.611 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.227566879999998, 2024-09-16T11:44:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:44:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:44:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:44:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:44:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:44:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:44:51.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:44:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:44:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:44:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:44:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:51.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:44:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:44:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:44:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:44:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:44:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:44:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.617 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.227544949000006, 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:44:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:44:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:44:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:44:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:44:51.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:44:51.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.227468060999996, 2024-09-16T11:44:51.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:44:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:44:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:44:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:44:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:44:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:44:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:44:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:44:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:44:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28187 4.17 e52/28 4:ee916bd9:::rbd_data.11c1ce740b46.0000000000000000:head v 52'195, mlcod=52'193)", 2024-09-16T11:44:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.056048675, 2024-09-16T11:44:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075854429999999999, 2024-09-16T11:44:51.629 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509349+0000", 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2300000000000001e-06 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513315+0000", 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039660590000000001 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513328+0000", 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.308e-05 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513334+0000", 2024-09-16T11:44:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:44:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513385+0000", 2024-09-16T11:44:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1347000000000002e-05 2024-09-16T11:44:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513407+0000", 2024-09-16T11:44:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1852999999999999e-05 2024-09-16T11:44:51.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516882+0000", 2024-09-16T11:44:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034749690000000001 2024-09-16T11:44:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516933+0000", 2024-09-16T11:44:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0952999999999998e-05 2024-09-16T11:44:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28959 4.8 e53/28 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head v 53'408, mlcod=53'406)", 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.533899619000003, 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088740420000000004, 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031498+0000", 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.652e-06 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033515+0000", 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020165930000000001 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033539+0000", 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4295999999999999e-05 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033548+0000", 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1239999999999994e-06 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033610+0000", 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1371999999999994e-05 2024-09-16T11:44:51.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033637+0000", 2024-09-16T11:44:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7098999999999999e-05 2024-09-16T11:44:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040341+0000", 2024-09-16T11:44:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067045819999999997 2024-09-16T11:44:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040371+0000", 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9323999999999999e-05 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32303 4.8 e53/28 4:1436ac82:::rbd_data.11c1cab081b3.0000000000000022:head v 53'417, mlcod=53'415)", 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.063473645000002, 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085565160000000001, 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:44:51.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-09-16T11:44:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501930+0000", 2024-09-16T11:44:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7649999999999996e-06 2024-09-16T11:44:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501946+0000", 2024-09-16T11:44:51.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5301000000000001e-05 2024-09-16T11:44:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501952+0000", 2024-09-16T11:44:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:44:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502100+0000", 2024-09-16T11:44:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014847600000000001 2024-09-16T11:44:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502120+0000", 2024-09-16T11:44:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9303000000000001e-05 2024-09-16T11:44:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510436+0000", 2024-09-16T11:44:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083166420000000008 2024-09-16T11:44:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510479+0000", 2024-09-16T11:44:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2979000000000003e-05 2024-09-16T11:44:51.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32306 4.1e e53/28 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head v 53'124, mlcod=53'122)", 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.063407650999999, 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084990900000000008, 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:44:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:44:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501994+0000", 2024-09-16T11:44:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:44:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502002+0000", 2024-09-16T11:44:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3839999999999997e-06 2024-09-16T11:44:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502007+0000", 2024-09-16T11:44:51.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6290000000000001e-06 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502454+0000", 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00044736800000000002 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502471+0000", 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6648999999999999e-05 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510455+0000", 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0079841909999999999 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510488+0000", 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2948999999999999e-05 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32307 4.8 e53/28 4:11c1c37d:::rbd_data.11c1cab081b3.0000000000000026:head v 53'418, mlcod=53'415)", 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:51.647 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.062652315999998, 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00776212, 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502750+0000", 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502760+0000", 2024-09-16T11:44:51.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:44:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502767+0000", 2024-09-16T11:44:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4219999999999997e-06 2024-09-16T11:44:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502802+0000", 2024-09-16T11:44:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5089999999999998e-05 2024-09-16T11:44:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502816+0000", 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4092e-05 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510483+0000", 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00766704 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510506+0000", 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3195999999999999e-05 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.651 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33583 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'416, mlcod=53'411)", 2024-09-16T11:44:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.354693906999998, 2024-09-16T11:44:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078132459999999994, 2024-09-16T11:44:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:44:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210703+0000", 2024-09-16T11:44:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:44:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210706+0000", 2024-09-16T11:44:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-06 2024-09-16T11:44:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210712+0000", 2024-09-16T11:44:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:44:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210715+0000", 2024-09-16T11:44:51.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:44:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210736+0000", 2024-09-16T11:44:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0293999999999999e-05 2024-09-16T11:44:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210746+0000", 2024-09-16T11:44:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0178e-05 2024-09-16T11:44:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218477+0000", 2024-09-16T11:44:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007731501 2024-09-16T11:44:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218516+0000", 2024-09-16T11:44:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8148000000000001e-05 2024-09-16T11:44:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.656 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33590 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'419, mlcod=53'415)", 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.343635765999998, 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082499879999999998, 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221761+0000", 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221765+0000", 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1590000000000003e-06 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221774+0000", 2024-09-16T11:44:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7490000000000004e-06 2024-09-16T11:44:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221778+0000", 2024-09-16T11:44:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0419999999999996e-06 2024-09-16T11:44:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221800+0000", 2024-09-16T11:44:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1926999999999999e-05 2024-09-16T11:44:51.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221810+0000", 2024-09-16T11:44:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0079e-05 2024-09-16T11:44:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.229980+0000", 2024-09-16T11:44:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081698469999999992 2024-09-16T11:44:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230010+0000", 2024-09-16T11:44:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0746e-05 2024-09-16T11:44:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33591 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'420, mlcod=53'415)", 2024-09-16T11:44:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.343074282000003, 2024-09-16T11:44:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007702576, 2024-09-16T11:44:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.661 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:44:51.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:44:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222326+0000", 2024-09-16T11:44:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:44:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222337+0000", 2024-09-16T11:44:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1008e-05 2024-09-16T11:44:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222342+0000", 2024-09-16T11:44:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0050000000000004e-06 2024-09-16T11:44:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222366+0000", 2024-09-16T11:44:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3696000000000001e-05 2024-09-16T11:44:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222385+0000", 2024-09-16T11:44:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-16T11:44:51.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230012+0000", 2024-09-16T11:44:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076270940000000001 2024-09-16T11:44:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230025+0000", 2024-09-16T11:44:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2112e-05 2024-09-16T11:44:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:55282 4.1e e73/28 4:785299e2:::rbd_data.11c11a49c2da.0000000000000001:head v 73'324, mlcod=73'322)", 2024-09-16T11:44:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:44:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.326377763, 2024-09-16T11:44:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071431890000000003, 2024-09-16T11:44:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:44:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:44:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239020+0000", 2024-09-16T11:44:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.049e-06 2024-09-16T11:44:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244238+0000", 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052183489999999997 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244252+0000", 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4246999999999999e-05 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244275+0000", 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3093999999999999e-05 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244314+0000", 2024-09-16T11:44:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8495000000000003e-05 2024-09-16T11:44:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244330+0000", 2024-09-16T11:44:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5917e-05 2024-09-16T11:44:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246121+0000", 2024-09-16T11:44:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017909239999999999 2024-09-16T11:44:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246162+0000", 2024-09-16T11:44:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1114e-05 2024-09-16T11:44:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:61651 4.11 e83/28 4:8f4a51c5:::rbd_data.11c175f8cab7.0000000000000014:head v 83'694, mlcod=81'692)", 2024-09-16T11:44:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:44:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.951958216000001, 2024-09-16T11:44:51.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.009466482, 2024-09-16T11:44:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:44:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:44:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613439+0000", 2024-09-16T11:44:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8400000000000003e-07 2024-09-16T11:44:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618867+0000", 2024-09-16T11:44:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054276059999999998 2024-09-16T11:44:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:51.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618883+0000", 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6265000000000001e-05 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618890+0000", 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2980000000000003e-06 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618950+0000", 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0025999999999998e-05 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618971+0000", 2024-09-16T11:44:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0458999999999998e-05 2024-09-16T11:44:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:44:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622869+0000", 2024-09-16T11:44:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038983450000000001 2024-09-16T11:44:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622905+0000", 2024-09-16T11:44:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599000000000003e-05 2024-09-16T11:44:51.674 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:51.675 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:51.675 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:51.676 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:51.674+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:51.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:51.774 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:51.774 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:51.774 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:51.775 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:51.777 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:51.774+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:51.783 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:51.878 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:51.876+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:51.881 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:51.881 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:51.881 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:51.881 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:51.881 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:51.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:51.979 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:51.977+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:51.988 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89607 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1327104~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:44:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.058206451, 2024-09-16T11:44:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.033363680999999999, 2024-09-16T11:44:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89607 2024-09-16T11:44:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:51.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:44:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:44:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:51.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929100+0000", 2024-09-16T11:44:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001870776 2024-09-16T11:44:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929104+0000", 2024-09-16T11:44:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1160000000000001e-06 2024-09-16T11:44:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929118+0000", 2024-09-16T11:44:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3891e-05 2024-09-16T11:44:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929126+0000", 2024-09-16T11:44:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6590000000000007e-06 2024-09-16T11:44:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960508+0000", 2024-09-16T11:44:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031381920000000001 2024-09-16T11:44:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:51.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960548+0000", 2024-09-16T11:44:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0734e-05 2024-09-16T11:44:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960593+0000", 2024-09-16T11:44:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4585000000000001e-05 2024-09-16T11:44:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89611 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1687552~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:51.993 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:44:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.056224491, 2024-09-16T11:44:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031799929999999997, 2024-09-16T11:44:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89611 2024-09-16T11:44:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:51.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:44:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:44:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929212+0000", 2024-09-16T11:44:51.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0800000000000001e-07 2024-09-16T11:44:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929213+0000", 2024-09-16T11:44:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6509999999999999e-06 2024-09-16T11:44:51.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929218+0000", 2024-09-16T11:44:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9899999999999997e-06 2024-09-16T11:44:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929221+0000", 2024-09-16T11:44:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1140000000000002e-06 2024-09-16T11:44:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960926+0000", 2024-09-16T11:44:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031704436000000003 2024-09-16T11:44:51.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960966+0000", 2024-09-16T11:44:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0055999999999997e-05 2024-09-16T11:44:51.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961011+0000", 2024-09-16T11:44:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5374999999999998e-05 2024-09-16T11:44:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.000 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89612 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1777664~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.056211282, 2024-09-16T11:44:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031893013999999997, 2024-09-16T11:44:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89612 2024-09-16T11:44:52.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6e-07 2024-09-16T11:44:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929226+0000", 2024-09-16T11:44:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5099999999999999e-06 2024-09-16T11:44:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929231+0000", 2024-09-16T11:44:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4889999999999997e-06 2024-09-16T11:44:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929234+0000", 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7410000000000001e-06 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961033+0000", 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031799279 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961071+0000", 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7759000000000002e-05 2024-09-16T11:44:52.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961118+0000", 2024-09-16T11:44:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6975999999999999e-05 2024-09-16T11:44:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89613 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1867776~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:52.005 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.056199143000001, 2024-09-16T11:44:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031996359000000002, 2024-09-16T11:44:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89613 2024-09-16T11:44:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.006 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8999999999999998e-07 2024-09-16T11:44:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929243+0000", 2024-09-16T11:44:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8719999999999999e-06 2024-09-16T11:44:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929249+0000", 2024-09-16T11:44:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6229999999999999e-06 2024-09-16T11:44:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929253+0000", 2024-09-16T11:44:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:44:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961145+0000", 2024-09-16T11:44:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031892648000000003 2024-09-16T11:44:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961188+0000", 2024-09-16T11:44:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2466999999999998e-05 2024-09-16T11:44:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961233+0000", 2024-09-16T11:44:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5049999999999997e-05 2024-09-16T11:44:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.012 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89615 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1957888~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:52.013 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.056179273, 2024-09-16T11:44:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032076532999999997, 2024-09-16T11:44:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.014 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89615 2024-09-16T11:44:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:52.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929259+0000", 2024-09-16T11:44:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7269999999999999e-06 2024-09-16T11:44:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929264+0000", 2024-09-16T11:44:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.524e-06 2024-09-16T11:44:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929267+0000", 2024-09-16T11:44:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.994e-06 2024-09-16T11:44:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961252+0000", 2024-09-16T11:44:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031985188999999997 2024-09-16T11:44:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961291+0000", 2024-09-16T11:44:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8263000000000003e-05 2024-09-16T11:44:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961333+0000", 2024-09-16T11:44:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2491999999999999e-05 2024-09-16T11:44:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89617 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 2048000~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.05616564, 2024-09-16T11:44:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032185244000000002, 2024-09-16T11:44:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89617 2024-09-16T11:44:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0499999999999999e-07 2024-09-16T11:44:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929272+0000", 2024-09-16T11:44:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.615e-06 2024-09-16T11:44:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929279+0000", 2024-09-16T11:44:52.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5459999999999997e-06 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929282+0000", 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.472e-06 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961361+0000", 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032078441999999999 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961401+0000", 2024-09-16T11:44:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0425e-05 2024-09-16T11:44:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961455+0000", 2024-09-16T11:44:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4438999999999998e-05 2024-09-16T11:44:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.025 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89637 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4235264~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:44:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.056145383, 2024-09-16T11:44:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032120463000000002, 2024-09-16T11:44:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89637 2024-09-16T11:44:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:44:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:44:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929291+0000", 2024-09-16T11:44:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:44:52.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929292+0000", 2024-09-16T11:44:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3120000000000001e-06 2024-09-16T11:44:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929295+0000", 2024-09-16T11:44:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3349999999999998e-06 2024-09-16T11:44:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929297+0000", 2024-09-16T11:44:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0549999999999998e-06 2024-09-16T11:44:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961346+0000", 2024-09-16T11:44:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032048920000000002 2024-09-16T11:44:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961379+0000", 2024-09-16T11:44:52.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2533000000000002e-05 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961411+0000", 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1921000000000002e-05 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89639 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4325376~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.056136304000001, 2024-09-16T11:44:52.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032194749000000002, 2024-09-16T11:44:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89639 2024-09-16T11:44:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:44:52.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:44:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929300+0000", 2024-09-16T11:44:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:44:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929301+0000", 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.018e-06 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929304+0000", 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8890000000000002e-06 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929306+0000", 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.883e-06 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961422+0000", 2024-09-16T11:44:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032116899999999997 2024-09-16T11:44:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961460+0000", 2024-09-16T11:44:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7265999999999999e-05 2024-09-16T11:44:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961494+0000", 2024-09-16T11:44:52.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4499000000000003e-05 2024-09-16T11:44:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89641 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4415488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:44:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.056128381000001, 2024-09-16T11:44:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032287719999999999, 2024-09-16T11:44:52.035 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89641 2024-09-16T11:44:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:44:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:44:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929308+0000", 2024-09-16T11:44:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1500000000000001e-07 2024-09-16T11:44:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929309+0000", 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.02e-06 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929312+0000", 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3000000000000002e-06 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929314+0000", 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0449999999999999e-06 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961521+0000", 2024-09-16T11:44:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032206835000000003 2024-09-16T11:44:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961561+0000", 2024-09-16T11:44:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0580000000000001e-05 2024-09-16T11:44:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961595+0000", 2024-09-16T11:44:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3725e-05 2024-09-16T11:44:52.039 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89643 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4505600~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.056119943000001, 2024-09-16T11:44:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032377892999999998, 2024-09-16T11:44:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.040 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89643 2024-09-16T11:44:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0100000000000001e-07 2024-09-16T11:44:52.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929317+0000", 2024-09-16T11:44:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.161e-06 2024-09-16T11:44:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929320+0000", 2024-09-16T11:44:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1580000000000001e-06 2024-09-16T11:44:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929322+0000", 2024-09-16T11:44:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9929999999999998e-06 2024-09-16T11:44:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961619+0000", 2024-09-16T11:44:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032296392 2024-09-16T11:44:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961659+0000", 2024-09-16T11:44:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9805000000000003e-05 2024-09-16T11:44:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961694+0000", 2024-09-16T11:44:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5182999999999998e-05 2024-09-16T11:44:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89644 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4243456~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.056111564, 2024-09-16T11:44:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032467912000000002, 2024-09-16T11:44:52.048 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89644 2024-09-16T11:44:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0100000000000001e-07 2024-09-16T11:44:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929325+0000", 2024-09-16T11:44:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9899999999999999e-07 2024-09-16T11:44:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929328+0000", 2024-09-16T11:44:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7539999999999998e-06 2024-09-16T11:44:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929330+0000", 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3690000000000001e-06 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961725+0000", 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032394088000000001 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961759+0000", 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4316999999999997e-05 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961792+0000", 2024-09-16T11:44:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3284000000000002e-05 2024-09-16T11:44:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.055 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89645 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4595712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.056103492, 2024-09-16T11:44:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032554313000000001, 2024-09-16T11:44:52.056 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89645 2024-09-16T11:44:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.99e-07 2024-09-16T11:44:52.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929333+0000", 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9299999999999996e-07 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929336+0000", 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8459999999999999e-06 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929340+0000", 2024-09-16T11:44:52.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6370000000000001e-06 2024-09-16T11:44:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961814+0000", 2024-09-16T11:44:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032474461000000003 2024-09-16T11:44:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961852+0000", 2024-09-16T11:44:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7877999999999999e-05 2024-09-16T11:44:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961887+0000", 2024-09-16T11:44:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4399e-05 2024-09-16T11:44:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89646 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4333568~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.056093855, 2024-09-16T11:44:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032641146000000003, 2024-09-16T11:44:52.061 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89646 2024-09-16T11:44:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.04e-07 2024-09-16T11:44:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929343+0000", 2024-09-16T11:44:52.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7000000000000003e-07 2024-09-16T11:44:52.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.064 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929346+0000", 2024-09-16T11:44:52.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7350000000000001e-06 2024-09-16T11:44:52.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929349+0000", 2024-09-16T11:44:52.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7030000000000002e-06 2024-09-16T11:44:52.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961916+0000", 2024-09-16T11:44:52.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032567634999999998 2024-09-16T11:44:52.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961950+0000", 2024-09-16T11:44:52.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3510000000000003e-05 2024-09-16T11:44:52.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961983+0000", 2024-09-16T11:44:52.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3389000000000002e-05 2024-09-16T11:44:52.067 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.067 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.067 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.067 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89647 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4685824~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.067 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:44:52.067 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.056085453, 2024-09-16T11:44:52.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032727342, 2024-09-16T11:44:52.067 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.067 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.068 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.068 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.068 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.068 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89647 2024-09-16T11:44:52.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.068 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.068 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.068 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:44:52.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:44:52.069 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931111+0000", 2024-09-16T11:44:52.069 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001760462 2024-09-16T11:44:52.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931114+0000", 2024-09-16T11:44:52.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.67e-06 2024-09-16T11:44:52.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931122+0000", 2024-09-16T11:44:52.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6059999999999997e-06 2024-09-16T11:44:52.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931126+0000", 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-06 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962005+0000", 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.030879125 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962044+0000", 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8779000000000003e-05 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962078+0000", 2024-09-16T11:44:52.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4010000000000001e-05 2024-09-16T11:44:52.072 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.072 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.072 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.072 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89656 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4784128~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.072 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:44:52.072 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.054235404, 2024-09-16T11:44:52.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031742976999999999, 2024-09-16T11:44:52.072 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.073 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.073 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.073 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.073 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.073 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89656 2024-09-16T11:44:52.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.073 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:44:52.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:44:52.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931201+0000", 2024-09-16T11:44:52.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0699999999999999e-07 2024-09-16T11:44:52.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931202+0000", 2024-09-16T11:44:52.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5900000000000005e-07 2024-09-16T11:44:52.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931204+0000", 2024-09-16T11:44:52.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8439999999999998e-06 2024-09-16T11:44:52.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931206+0000", 2024-09-16T11:44:52.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.871e-06 2024-09-16T11:44:52.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962877+0000", 2024-09-16T11:44:52.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031670831000000003 2024-09-16T11:44:52.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962910+0000", 2024-09-16T11:44:52.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3405999999999998e-05 2024-09-16T11:44:52.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962943+0000", 2024-09-16T11:44:52.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2858999999999998e-05 2024-09-16T11:44:52.076 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.077 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.077 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.077 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89657 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5136384~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.077 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:52.077 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.054227593, 2024-09-16T11:44:52.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031827938, 2024-09-16T11:44:52.077 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.077 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.077 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.077 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.077 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89657 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:52.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.28e-07 2024-09-16T11:44:52.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931209+0000", 2024-09-16T11:44:52.080 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9099999999999991e-07 2024-09-16T11:44:52.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.080 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931213+0000", 2024-09-16T11:44:52.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7639999999999999e-06 2024-09-16T11:44:52.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931217+0000", 2024-09-16T11:44:52.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4759999999999998e-06 2024-09-16T11:44:52.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962964+0000", 2024-09-16T11:44:52.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031747598000000002 2024-09-16T11:44:52.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.083 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963003+0000", 2024-09-16T11:44:52.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8287000000000001e-05 2024-09-16T11:44:52.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.083 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.083 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.083 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963036+0000", 2024-09-16T11:44:52.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3593999999999997e-05 2024-09-16T11:44:52.083 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.083 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.084 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.084 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89658 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4874240~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.084 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:52.084 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.054200098000001, 2024-09-16T11:44:52.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031905067000000002, 2024-09-16T11:44:52.084 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.084 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.084 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.085 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.085 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.085 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89658 2024-09-16T11:44:52.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.085 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:52.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4700000000000001e-07 2024-09-16T11:44:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931238+0000", 2024-09-16T11:44:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:44:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931245+0000", 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.64e-06 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931248+0000", 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5719999999999999e-06 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963066+0000", 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031817326999999999 2024-09-16T11:44:52.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963107+0000", 2024-09-16T11:44:52.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1220000000000002e-05 2024-09-16T11:44:52.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963141+0000", 2024-09-16T11:44:52.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3896999999999997e-05 2024-09-16T11:44:52.089 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.089 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.089 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.090 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89659 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4964352~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.090 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:52.090 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.054183796, 2024-09-16T11:44:52.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031984460999999999, 2024-09-16T11:44:52.090 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.090 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.091 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.091 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.091 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.091 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89659 2024-09-16T11:44:52.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.091 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:52.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:52.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.093 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.093 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:52.093 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:52.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.093 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.093 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931254+0000", 2024-09-16T11:44:52.093 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.776e-06 2024-09-16T11:44:52.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.093 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.093 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931262+0000", 2024-09-16T11:44:52.093 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1680000000000004e-06 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931266+0000", 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.732e-06 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963165+0000", 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031898693999999998 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963202+0000", 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7517e-05 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963236+0000", 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4230000000000003e-05 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89660 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5226496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:52.095 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.054165776, 2024-09-16T11:44:52.096 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032062368000000001, 2024-09-16T11:44:52.096 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.096 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.096 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.096 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.096 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.096 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89660 2024-09-16T11:44:52.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.096 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.096 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.096 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:52.097 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.097 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.097 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:52.097 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.097 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.097 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:52.097 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.39e-07 2024-09-16T11:44:52.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.098 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.098 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931272+0000", 2024-09-16T11:44:52.098 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:44:52.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.098 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.098 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931279+0000", 2024-09-16T11:44:52.098 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:44:52.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.099 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.099 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931282+0000", 2024-09-16T11:44:52.099 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.721e-06 2024-09-16T11:44:52.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.099 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.099 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963266+0000", 2024-09-16T11:44:52.100 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031983971999999999 2024-09-16T11:44:52.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.100 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.100 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963300+0000", 2024-09-16T11:44:52.100 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3318000000000002e-05 2024-09-16T11:44:52.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.101 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.101 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963332+0000", 2024-09-16T11:44:52.101 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2768000000000002e-05 2024-09-16T11:44:52.101 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.101 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.101 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.101 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89661 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5054464~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:52.101 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:44:52.101 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.054149352, 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032139394000000002, 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89661 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933104+0000", 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00181734 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933107+0000", 2024-09-16T11:44:52.103 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5140000000000002e-06 2024-09-16T11:44:52.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.104 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.104 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933115+0000", 2024-09-16T11:44:52.104 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:44:52.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.104 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.104 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933124+0000", 2024-09-16T11:44:52.104 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8910000000000001e-06 2024-09-16T11:44:52.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.104 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963354+0000", 2024-09-16T11:44:52.105 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.030230825999999999 2024-09-16T11:44:52.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963392+0000", 2024-09-16T11:44:52.105 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7765000000000002e-05 2024-09-16T11:44:52.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963426+0000", 2024-09-16T11:44:52.105 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3603000000000003e-05 2024-09-16T11:44:52.105 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.106 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.106 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.106 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.106 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.106 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:52.107 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:52.108 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:52.105+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:52.205 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:52.205 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:52.205 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:52.205 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:52.207 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:52.205+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:52.213 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:52.280 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:44:52.281 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:44:52.281 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T11:44:52.281 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-16T11:44:52.311 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:52.311 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:52.311 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:52.311 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:52.311 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:52.312 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:52.309+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:52.321 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:52.410 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:52.408+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:52.462 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:52.462 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:52.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:52.462 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:52.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.462 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19364 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1171456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.946246721000001, 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049001805000000002, 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19364 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:52.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:52.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:52.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:44:52.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513280+0000", 2024-09-16T11:44:52.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7090000000000001e-06 2024-09-16T11:44:52.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513286+0000", 2024-09-16T11:44:52.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5840000000000002e-06 2024-09-16T11:44:52.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513290+0000", 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3680000000000004e-06 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562121+0000", 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048831324000000002 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562195+0000", 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3415000000000001e-05 2024-09-16T11:44:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562280+0000", 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4975999999999994e-05 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19365 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1490944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.946226981999999, 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049335349000000001, 2024-09-16T11:44:52.467 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19365 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513300+0000", 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3020000000000002e-06 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.470 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.470 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513306+0000", 2024-09-16T11:44:52.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8780000000000003e-06 2024-09-16T11:44:52.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.470 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.470 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513310+0000", 2024-09-16T11:44:52.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1309999999999999e-06 2024-09-16T11:44:52.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.470 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.470 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562322+0000", 2024-09-16T11:44:52.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049011635999999997 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562544+0000", 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022168000000000001 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562633+0000", 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9350000000000003e-05 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19366 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1916928~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.946210049999998, 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049865073000000003, 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19366 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:52.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:52.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515102+0000", 2024-09-16T11:44:52.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017877419999999999 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515106+0000", 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4450000000000001e-06 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515113+0000", 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7680000000000008e-06 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515117+0000", 2024-09-16T11:44:52.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:44:52.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562756+0000", 2024-09-16T11:44:52.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047638579 2024-09-16T11:44:52.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563093+0000", 2024-09-16T11:44:52.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000337516 2024-09-16T11:44:52.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563180+0000", 2024-09-16T11:44:52.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6420000000000003e-05 2024-09-16T11:44:52.476 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.476 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.476 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.476 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19371 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3194880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.476 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:52.476 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.944312842999999, 2024-09-16T11:44:52.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048948356999999998, 2024-09-16T11:44:52.476 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.477 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.477 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.477 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.477 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.477 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19371 2024-09-16T11:44:52.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.477 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:52.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515214+0000", 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5969999999999999e-06 2024-09-16T11:44:52.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515219+0000", 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4559999999999999e-06 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515223+0000", 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563991+0000", 2024-09-16T11:44:52.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048768911999999998 2024-09-16T11:44:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564063+0000", 2024-09-16T11:44:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2010999999999999e-05 2024-09-16T11:44:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564160+0000", 2024-09-16T11:44:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6617e-05 2024-09-16T11:44:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.480 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19372 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3620864~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.944298625999998, 2024-09-16T11:44:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049130502, 2024-09-16T11:44:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19372 2024-09-16T11:44:52.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515228+0000", 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515233+0000", 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515237+0000", 2024-09-16T11:44:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.27e-06 2024-09-16T11:44:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564199+0000", 2024-09-16T11:44:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048961787999999999 2024-09-16T11:44:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564275+0000", 2024-09-16T11:44:52.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6001000000000006e-05 2024-09-16T11:44:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564356+0000", 2024-09-16T11:44:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1941999999999996e-05 2024-09-16T11:44:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19373 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3940352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.485 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.944284226000001, 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049289588000000002, 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19373 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:52.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515241+0000", 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515242+0000", 2024-09-16T11:44:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.528e-06 2024-09-16T11:44:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515248+0000", 2024-09-16T11:44:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515251+0000", 2024-09-16T11:44:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2480000000000001e-06 2024-09-16T11:44:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564388+0000", 2024-09-16T11:44:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049137313000000002 2024-09-16T11:44:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564455+0000", 2024-09-16T11:44:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7049000000000001e-05 2024-09-16T11:44:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564530+0000", 2024-09-16T11:44:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4626e-05 2024-09-16T11:44:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19374 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 4046848~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.944269822999999, 2024-09-16T11:44:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049457133, 2024-09-16T11:44:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.490 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19374 2024-09-16T11:44:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515257+0000", 2024-09-16T11:44:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5740000000000001e-06 2024-09-16T11:44:52.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515262+0000", 2024-09-16T11:44:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2399999999999998e-06 2024-09-16T11:44:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515265+0000", 2024-09-16T11:44:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3909999999999998e-06 2024-09-16T11:44:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564555+0000", 2024-09-16T11:44:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049290002999999999 2024-09-16T11:44:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564628+0000", 2024-09-16T11:44:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2559999999999996e-05 2024-09-16T11:44:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564712+0000", 2024-09-16T11:44:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4040000000000002e-05 2024-09-16T11:44:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19375 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3301376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.944255679999998, 2024-09-16T11:44:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049606631999999998, 2024-09-16T11:44:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.495 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19375 2024-09-16T11:44:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:44:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515271+0000", 2024-09-16T11:44:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5230000000000001e-06 2024-09-16T11:44:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515276+0000", 2024-09-16T11:44:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4650000000000002e-06 2024-09-16T11:44:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515279+0000", 2024-09-16T11:44:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0630000000000002e-06 2024-09-16T11:44:52.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564738+0000", 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049459001000000002 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564780+0000", 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1693999999999998e-05 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564876+0000", 2024-09-16T11:44:52.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5571999999999999e-05 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19376 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3727360~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.944241767999998, 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049720590000000002, 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.500 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19376 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517103+0000", 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001820276 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517108+0000", 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5410000000000002e-06 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517120+0000", 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2092e-05 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517125+0000", 2024-09-16T11:44:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6829999999999996e-06 2024-09-16T11:44:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564870+0000", 2024-09-16T11:44:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047745040000000002 2024-09-16T11:44:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564921+0000", 2024-09-16T11:44:52.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0538999999999998e-05 2024-09-16T11:44:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565003+0000", 2024-09-16T11:44:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2418999999999998e-05 2024-09-16T11:44:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19492 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 434176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.504 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.942312911999998, 2024-09-16T11:44:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048926948999999997, 2024-09-16T11:44:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19492 2024-09-16T11:44:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3099999999999999e-07 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517213+0000", 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6599999999999994e-07 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517216+0000", 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.706e-06 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517218+0000", 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.035e-06 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565959+0000", 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048741380000000001 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566036+0000", 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7362999999999998e-05 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566139+0000", 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102268 2024-09-16T11:44:52.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19493 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 860160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.942291241, 2024-09-16T11:44:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049086604999999998, 2024-09-16T11:44:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.509 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19493 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517234+0000", 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.37e-07 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517236+0000", 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7150000000000001e-06 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517243+0000", 2024-09-16T11:44:52.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9489999999999999e-06 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517246+0000", 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.636e-06 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566163+0000", 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048917284999999998 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566237+0000", 2024-09-16T11:44:52.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3196999999999997e-05 2024-09-16T11:44:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566320+0000", 2024-09-16T11:44:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3286000000000002e-05 2024-09-16T11:44:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19494 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1179648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:52.513 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.942270624999999, 2024-09-16T11:44:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049259562999999999, 2024-09-16T11:44:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19494 2024-09-16T11:44:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:52.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517256+0000", 2024-09-16T11:44:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.888e-06 2024-09-16T11:44:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517265+0000", 2024-09-16T11:44:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8629999999999997e-06 2024-09-16T11:44:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517269+0000", 2024-09-16T11:44:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2039999999999999e-06 2024-09-16T11:44:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566352+0000", 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049083107000000001 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566429+0000", 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7025000000000002e-05 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566514+0000", 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4121000000000003e-05 2024-09-16T11:44:52.517 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19495 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1286144~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.942251202000001, 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049492880000000003, 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19495 2024-09-16T11:44:52.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517274+0000", 2024-09-16T11:44:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:44:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517276+0000", 2024-09-16T11:44:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8950000000000001e-06 2024-09-16T11:44:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517281+0000", 2024-09-16T11:44:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3879999999999999e-06 2024-09-16T11:44:52.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517285+0000", 2024-09-16T11:44:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5930000000000002e-06 2024-09-16T11:44:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566579+0000", 2024-09-16T11:44:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049294175000000003 2024-09-16T11:44:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566670+0000", 2024-09-16T11:44:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1096999999999999e-05 2024-09-16T11:44:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566766+0000", 2024-09-16T11:44:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6355999999999997e-05 2024-09-16T11:44:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19496 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1605632~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.942236117, 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049632981999999999, 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19496 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.523 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517289+0000", 2024-09-16T11:44:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:52.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517290+0000", 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.581e-06 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517300+0000", 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517304+0000", 2024-09-16T11:44:52.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566789+0000", 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049485102000000003 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566843+0000", 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3937000000000002e-05 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566921+0000", 2024-09-16T11:44:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8397000000000002e-05 2024-09-16T11:44:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19497 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 114688~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.942216422000001, 2024-09-16T11:44:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049761367000000001, 2024-09-16T11:44:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.527 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19497 2024-09-16T11:44:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.528 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519116+0000", 2024-09-16T11:44:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018079439999999999 2024-09-16T11:44:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519120+0000", 2024-09-16T11:44:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2760000000000002e-06 2024-09-16T11:44:52.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519134+0000", 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3468999999999999e-05 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519140+0000", 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.516e-06 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566943+0000", 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047802561 2024-09-16T11:44:52.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566982+0000", 2024-09-16T11:44:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9269e-05 2024-09-16T11:44:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567070+0000", 2024-09-16T11:44:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7331999999999997e-05 2024-09-16T11:44:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19504 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 966656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.940287935000001, 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049045863000000002, 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19504 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:52.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519239+0000", 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.962e-06 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519244+0000", 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5480000000000001e-06 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519248+0000", 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.641e-06 2024-09-16T11:44:52.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568150+0000", 2024-09-16T11:44:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048901864000000003 2024-09-16T11:44:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568208+0000", 2024-09-16T11:44:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8399000000000001e-05 2024-09-16T11:44:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568283+0000", 2024-09-16T11:44:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4105999999999998e-05 2024-09-16T11:44:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19505 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1392640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.940272630000003, 2024-09-16T11:44:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049202304000000002, 2024-09-16T11:44:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.536 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19505 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:52.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519254+0000", 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.761e-06 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519257+0000", 2024-09-16T11:44:52.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.343e-06 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519259+0000", 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0559999999999999e-06 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568307+0000", 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049047445000000002 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568382+0000", 2024-09-16T11:44:52.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5574999999999994e-05 2024-09-16T11:44:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568454+0000", 2024-09-16T11:44:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1799999999999997e-05 2024-09-16T11:44:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19506 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1712128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:52.540 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.940262592000003, 2024-09-16T11:44:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049341532, 2024-09-16T11:44:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19506 2024-09-16T11:44:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:52.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521102+0000", 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018404700000000001 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521106+0000", 2024-09-16T11:44:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8820000000000003e-06 2024-09-16T11:44:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521119+0000", 2024-09-16T11:44:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2546e-05 2024-09-16T11:44:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521125+0000", 2024-09-16T11:44:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0249999999999999e-06 2024-09-16T11:44:52.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568484+0000", 2024-09-16T11:44:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047359120999999997 2024-09-16T11:44:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568542+0000", 2024-09-16T11:44:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8360999999999999e-05 2024-09-16T11:44:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568604+0000", 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1126999999999999e-05 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19515 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1499136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 56.938270307000003, 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048949431000000002, 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.545 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19515 2024-09-16T11:44:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:52.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523130+0000", 2024-09-16T11:44:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00187604 2024-09-16T11:44:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523134+0000", 2024-09-16T11:44:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:44:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523144+0000", 2024-09-16T11:44:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2010000000000005e-06 2024-09-16T11:44:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523149+0000", 2024-09-16T11:44:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0039999999999999e-06 2024-09-16T11:44:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570113+0000", 2024-09-16T11:44:52.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.046964517999999997 2024-09-16T11:44:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570157+0000", 2024-09-16T11:44:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4056e-05 2024-09-16T11:44:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570204+0000", 2024-09-16T11:44:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6591999999999997e-05 2024-09-16T11:44:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.549 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:44111 4.8 4:17f3f47e:::rbd_data.11c1e170f5f0.000000000000002f:head [delete] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:44:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 42.476290026000001, 2024-09-16T11:44:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93903426400000001, 2024-09-16T11:44:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.550 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 44111 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7199999999999998e-07 2024-09-16T11:44:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983239+0000", 2024-09-16T11:44:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9879999999999998e-06 2024-09-16T11:44:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983248+0000", 2024-09-16T11:44:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.371e-06 2024-09-16T11:44:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983253+0000", 2024-09-16T11:44:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1150000000000002e-06 2024-09-16T11:44:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983303+0000", 2024-09-16T11:44:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0380000000000002e-05 2024-09-16T11:44:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:44:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983311+0000", 2024-09-16T11:44:52.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7519999999999996e-06 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920618+0000", 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93730672000000004 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920730+0000", 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011265900000000001 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:52.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920842+0000", 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111638 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.921503+0000", 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066074299999999999 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922208+0000", 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00070492899999999997 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922233+0000", 2024-09-16T11:44:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5922999999999999e-05 2024-09-16T11:44:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922269+0000", 2024-09-16T11:44:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5373999999999997e-05 2024-09-16T11:44:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.556 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.556 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:52.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:52.558 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:52.555+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:52.564 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:44:52.564 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:44:52.564 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.23071 -> 3.23071 (expected baseline 3.13748) 2024-09-16T11:44:52.564 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:44:52.564 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 18 -> 18 pgs (1.6 -> 1.6 * mean) 2024-09-16T11:44:52.564 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:52.564 INFO:teuthology.orchestra.run.smithi070.stdout:oload 110 2024-09-16T11:44:52.564 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 1 2024-09-16T11:44:52.564 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:44:52.565 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0033 2024-09-16T11:44:52.565 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0036 2024-09-16T11:44:52.565 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:44:52.578 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T11:44:52.579 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-16T11:44:52.659 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:52.656+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:52.665 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:52.665 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:52.666 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:52.666 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:52.674 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:52.760 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:52.758+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:52.775 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:52.775 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:52.775 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:52.775 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:52.775 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:52.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:52.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:44:52.861 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:52.859+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:52.884 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:52.885 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:52.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:52.885 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:52.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.885 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16853 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3375104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.885 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:52.885 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.534606330000003, 2024-09-16T11:44:52.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027791907000000001, 2024-09-16T11:44:52.885 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.885 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.885 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.886 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.886 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.886 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16853 2024-09-16T11:44:52.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.886 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:52.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:52.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347568+0000", 2024-09-16T11:44:52.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:44:52.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347570+0000", 2024-09-16T11:44:52.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:44:52.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347588+0000", 2024-09-16T11:44:52.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7914000000000002e-05 2024-09-16T11:44:52.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347592+0000", 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3150000000000002e-06 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375199+0000", 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027606065999999999 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375276+0000", 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7527000000000005e-05 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.890 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375359+0000", 2024-09-16T11:44:52.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3238000000000005e-05 2024-09-16T11:44:52.890 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.890 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.890 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.890 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16854 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.890 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:52.890 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.534576633, 2024-09-16T11:44:52.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027945148999999999, 2024-09-16T11:44:52.890 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.890 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.891 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.891 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.891 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.891 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16854 2024-09-16T11:44:52.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.891 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:52.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:52.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347598+0000", 2024-09-16T11:44:52.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:44:52.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347600+0000", 2024-09-16T11:44:52.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:44:52.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347606+0000", 2024-09-16T11:44:52.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0580000000000003e-06 2024-09-16T11:44:52.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347609+0000", 2024-09-16T11:44:52.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:44:52.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375386+0000", 2024-09-16T11:44:52.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027776809 2024-09-16T11:44:52.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375461+0000", 2024-09-16T11:44:52.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4868000000000002e-05 2024-09-16T11:44:52.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375542+0000", 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1253000000000003e-05 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16855 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 4120576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.534560294000002, 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028111094999999999, 2024-09-16T11:44:52.895 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16855 2024-09-16T11:44:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:52.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347614+0000", 2024-09-16T11:44:52.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:44:52.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347616+0000", 2024-09-16T11:44:52.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:44:52.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347623+0000", 2024-09-16T11:44:52.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0099999999999998e-06 2024-09-16T11:44:52.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347626+0000", 2024-09-16T11:44:52.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2140000000000001e-06 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375569+0000", 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027942649999999999 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375643+0000", 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3949999999999995e-05 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375725+0000", 2024-09-16T11:44:52.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1992999999999999e-05 2024-09-16T11:44:52.900 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.900 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.900 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.900 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16856 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3801088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.900 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:52.900 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.534538222999998, 2024-09-16T11:44:52.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028268272000000001, 2024-09-16T11:44:52.900 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.901 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.901 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.901 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.901 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.901 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16856 2024-09-16T11:44:52.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.902 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:52.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:52.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347636+0000", 2024-09-16T11:44:52.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:44:52.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347638+0000", 2024-09-16T11:44:52.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.86e-06 2024-09-16T11:44:52.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347644+0000", 2024-09-16T11:44:52.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1739999999999997e-06 2024-09-16T11:44:52.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347656+0000", 2024-09-16T11:44:52.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2198000000000001e-05 2024-09-16T11:44:52.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375751+0000", 2024-09-16T11:44:52.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028094507000000001 2024-09-16T11:44:52.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375824+0000", 2024-09-16T11:44:52.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3178999999999998e-05 2024-09-16T11:44:52.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.909 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.909 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375904+0000", 2024-09-16T11:44:52.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9985000000000001e-05 2024-09-16T11:44:52.909 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.909 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.910 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.910 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16857 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.910 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:52.910 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.534511721000001, 2024-09-16T11:44:52.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028275403000000001, 2024-09-16T11:44:52.911 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.911 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:52.912 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.912 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.912 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.912 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16857 2024-09-16T11:44:52.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.912 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:52.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.914 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:52.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:52.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.46e-07 2024-09-16T11:44:52.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347666+0000", 2024-09-16T11:44:52.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8739999999999997e-06 2024-09-16T11:44:52.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347672+0000", 2024-09-16T11:44:52.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1059999999999996e-06 2024-09-16T11:44:52.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347676+0000", 2024-09-16T11:44:52.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:44:52.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.918 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.918 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375930+0000", 2024-09-16T11:44:52.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028254042 2024-09-16T11:44:52.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375937+0000", 2024-09-16T11:44:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5519999999999999e-06 2024-09-16T11:44:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16858 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.534494076000001, 2024-09-16T11:44:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028284592000000001, 2024-09-16T11:44:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16858 2024-09-16T11:44:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:44:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347683+0000", 2024-09-16T11:44:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6809999999999998e-06 2024-09-16T11:44:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347689+0000", 2024-09-16T11:44:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347695+0000", 2024-09-16T11:44:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0020000000000003e-06 2024-09-16T11:44:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375958+0000", 2024-09-16T11:44:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028263870999999999 2024-09-16T11:44:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375964+0000", 2024-09-16T11:44:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8209999999999998e-06 2024-09-16T11:44:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16992 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.532919388000003, 2024-09-16T11:44:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027704776, 2024-09-16T11:44:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16992 2024-09-16T11:44:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349255+0000", 2024-09-16T11:44:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:44:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349260+0000", 2024-09-16T11:44:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5589999999999996e-06 2024-09-16T11:44:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349267+0000", 2024-09-16T11:44:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4019999999999999e-06 2024-09-16T11:44:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349270+0000", 2024-09-16T11:44:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6780000000000002e-06 2024-09-16T11:44:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376953+0000", 2024-09-16T11:44:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682933 2024-09-16T11:44:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376959+0000", 2024-09-16T11:44:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:44:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16993 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.532898062999998, 2024-09-16T11:44:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027707908999999999, 2024-09-16T11:44:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16993 2024-09-16T11:44:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349278+0000", 2024-09-16T11:44:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.629e-06 2024-09-16T11:44:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349291+0000", 2024-09-16T11:44:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614e-05 2024-09-16T11:44:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349295+0000", 2024-09-16T11:44:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1189999999999999e-06 2024-09-16T11:44:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376978+0000", 2024-09-16T11:44:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682445 2024-09-16T11:44:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376984+0000", 2024-09-16T11:44:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7590000000000003e-06 2024-09-16T11:44:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16994 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.532874051999997, 2024-09-16T11:44:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027708753999999999, 2024-09-16T11:44:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16994 2024-09-16T11:44:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:44:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349302+0000", 2024-09-16T11:44:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.722e-06 2024-09-16T11:44:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349308+0000", 2024-09-16T11:44:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:44:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349312+0000", 2024-09-16T11:44:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5310000000000002e-06 2024-09-16T11:44:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377003+0000", 2024-09-16T11:44:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027691308000000001 2024-09-16T11:44:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377008+0000", 2024-09-16T11:44:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4550000000000003e-06 2024-09-16T11:44:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16999 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.532731173999998, 2024-09-16T11:44:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027705270000000001, 2024-09-16T11:44:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16999 2024-09-16T11:44:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.46e-07 2024-09-16T11:44:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349445+0000", 2024-09-16T11:44:52.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.637e-06 2024-09-16T11:44:52.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349451+0000", 2024-09-16T11:44:52.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9959999999999999e-06 2024-09-16T11:44:52.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349454+0000", 2024-09-16T11:44:52.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.647e-06 2024-09-16T11:44:52.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377142+0000", 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027687942 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377148+0000", 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7019999999999998e-06 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.964 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17000 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.965 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:52.965 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.532715666999998, 2024-09-16T11:44:52.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.02771881, 2024-09-16T11:44:52.965 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.965 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:52.965 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.965 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.965 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.965 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17000 2024-09-16T11:44:52.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.965 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0600000000000001e-07 2024-09-16T11:44:52.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349460+0000", 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.736e-06 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349466+0000", 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2419999999999997e-06 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349470+0000", 2024-09-16T11:44:52.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5480000000000002e-06 2024-09-16T11:44:52.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377171+0000", 2024-09-16T11:44:52.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027701282000000001 2024-09-16T11:44:52.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377177+0000", 2024-09-16T11:44:52.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5960000000000003e-06 2024-09-16T11:44:52.968 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.968 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.968 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17001 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.532699917000002, 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027739243, 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17001 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.969 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:52.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:44:52.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349476+0000", 2024-09-16T11:44:52.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T11:44:52.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349482+0000", 2024-09-16T11:44:52.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2389999999999999e-06 2024-09-16T11:44:52.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349486+0000", 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5980000000000001e-06 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377206+0000", 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027720037999999999 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377213+0000", 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3440000000000003e-06 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.972 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17433 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 5898240~4888] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.496736263000003, 2024-09-16T11:44:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027709398, 2024-09-16T11:44:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.973 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17433 2024-09-16T11:44:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385438+0000", 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385439+0000", 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5120000000000001e-06 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385445+0000", 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4e-06 2024-09-16T11:44:52.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385448+0000", 2024-09-16T11:44:52.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1669999999999999e-06 2024-09-16T11:44:52.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412916+0000", 2024-09-16T11:44:52.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027468137 2024-09-16T11:44:52.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412988+0000", 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1606000000000005e-05 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413147+0000", 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015924299999999999 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17434 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6004736~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:52.977 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.496721673000003, 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027777461, 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17434 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:52.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:52.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:52.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385454+0000", 2024-09-16T11:44:52.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6729999999999999e-06 2024-09-16T11:44:52.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385459+0000", 2024-09-16T11:44:52.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4079999999999997e-06 2024-09-16T11:44:52.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385463+0000", 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.269e-06 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413157+0000", 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027694611000000001 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413193+0000", 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5673999999999998e-05 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413230+0000", 2024-09-16T11:44:52.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6454e-05 2024-09-16T11:44:52.982 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.982 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.982 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.982 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17435 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6324224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.982 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:52.982 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.496707520000001, 2024-09-16T11:44:52.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929059999999999, 2024-09-16T11:44:52.982 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17435 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385467+0000", 2024-09-16T11:44:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385468+0000", 2024-09-16T11:44:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6300000000000001e-06 2024-09-16T11:44:52.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385474+0000", 2024-09-16T11:44:52.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1299999999999998e-06 2024-09-16T11:44:52.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385478+0000", 2024-09-16T11:44:52.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2880000000000001e-06 2024-09-16T11:44:52.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413257+0000", 2024-09-16T11:44:52.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027779608000000001 2024-09-16T11:44:52.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413331+0000", 2024-09-16T11:44:52.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3639000000000004e-05 2024-09-16T11:44:52.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413395+0000", 2024-09-16T11:44:52.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4424999999999994e-05 2024-09-16T11:44:52.986 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.987 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.987 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.987 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17436 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7176192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.987 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:52.987 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.496691857999998, 2024-09-16T11:44:52.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028016718999999999, 2024-09-16T11:44:52.987 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.987 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.987 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.987 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.988 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.988 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17436 2024-09-16T11:44:52.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.988 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:52.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:52.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:52.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:52.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385484+0000", 2024-09-16T11:44:52.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5179999999999999e-06 2024-09-16T11:44:52.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385489+0000", 2024-09-16T11:44:52.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7049999999999996e-06 2024-09-16T11:44:52.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385493+0000", 2024-09-16T11:44:52.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T11:44:52.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413422+0000", 2024-09-16T11:44:52.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929611999999999 2024-09-16T11:44:52.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413462+0000", 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9585000000000001e-05 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413499+0000", 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6644999999999999e-05 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.992 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17437 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6750208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.992 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:52.992 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.496677443000003, 2024-09-16T11:44:52.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028167168999999999, 2024-09-16T11:44:52.992 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.992 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.992 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.992 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.992 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.992 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17437 2024-09-16T11:44:52.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.992 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:52.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:52.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385497+0000", 2024-09-16T11:44:52.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:44:52.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385498+0000", 2024-09-16T11:44:52.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.592e-06 2024-09-16T11:44:52.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385504+0000", 2024-09-16T11:44:52.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4090000000000002e-06 2024-09-16T11:44:52.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385513+0000", 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4460000000000007e-06 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413526+0000", 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028012866000000001 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413601+0000", 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5388000000000004e-05 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:52.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413663+0000", 2024-09-16T11:44:52.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.211e-05 2024-09-16T11:44:52.996 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.996 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:52.996 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:52.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.996 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17438 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7495680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:52.996 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:52.996 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.496654683999999, 2024-09-16T11:44:52.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028224439, 2024-09-16T11:44:52.996 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17438 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:52.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:52.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:52.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:52.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:52.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4900000000000001e-07 2024-09-16T11:44:52.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:52.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385521+0000", 2024-09-16T11:44:52.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:44:52.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385526+0000", 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2680000000000002e-06 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385529+0000", 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2040000000000002e-06 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413674+0000", 2024-09-16T11:44:52.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028144226000000001 2024-09-16T11:44:53.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:53.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413710+0000", 2024-09-16T11:44:53.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6665000000000002e-05 2024-09-16T11:44:53.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:53.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413743+0000", 2024-09-16T11:44:53.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3155999999999999e-05 2024-09-16T11:44:53.000 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:53.000 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:53.000 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17439 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7921664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.496593529999998, 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028326131000000001, 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17439 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.001 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:53.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:53.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:53.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:53.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:53.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:53.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:53.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:53.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385581+0000", 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9299999999999998e-07 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385583+0000", 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6220000000000001e-06 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385592+0000", 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5220000000000007e-06 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:53.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385595+0000", 2024-09-16T11:44:53.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8669999999999998e-06 2024-09-16T11:44:53.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:53.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413771+0000", 2024-09-16T11:44:53.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028175875999999999 2024-09-16T11:44:53.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:53.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413845+0000", 2024-09-16T11:44:53.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4382999999999996e-05 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413906+0000", 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1367999999999999e-05 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17440 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 57.496576322000003, 2024-09-16T11:44:53.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028391117, 2024-09-16T11:44:53.006 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:53.006 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:53.006 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:53.006 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:53.006 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:53.006 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17440 2024-09-16T11:44:53.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.006 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:53.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:53.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:53.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:53.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385598+0000", 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385603+0000", 2024-09-16T11:44:53.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:44:53.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:53.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385607+0000", 2024-09-16T11:44:53.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4869999999999998e-06 2024-09-16T11:44:53.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:53.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385609+0000", 2024-09-16T11:44:53.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2759999999999999e-06 2024-09-16T11:44:53.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:53.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413916+0000", 2024-09-16T11:44:53.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028307253000000001 2024-09-16T11:44:53.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:53.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413955+0000", 2024-09-16T11:44:53.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8652000000000002e-05 2024-09-16T11:44:53.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:53.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:53.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:53.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413989+0000", 2024-09-16T11:44:53.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3587999999999998e-05 2024-09-16T11:44:53.010 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:53.010 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:53.011 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:53.011 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:53.011 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:53.011 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:53.012 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:44:53.012 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:44:53.012 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.23071 -> 3.23071 (expected baseline 3.13748) 2024-09-16T11:44:53.012 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:44:53.012 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 18 -> 18 pgs (1.6 -> 1.6 * mean) 2024-09-16T11:44:53.012 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:53.013 INFO:teuthology.orchestra.run.smithi070.stdout:oload 110 2024-09-16T11:44:53.013 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 1 2024-09-16T11:44:53.013 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:44:53.013 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0033 2024-09-16T11:44:53.013 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0036 2024-09-16T11:44:53.013 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:44:53.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:53.015 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T11:44:53.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-16T11:44:53.015 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:53.016 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:53.016 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:53.016 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:53.017 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:53.017 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:53.017 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:53.018 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:53.010+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:53.018 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:53.018 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:53.018 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:53.034 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:53.034 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:53.050 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:53.050 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:53.066 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:53.119 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:53.119 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:53.120 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:53.117+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:53.129 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:53.214 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:53.212+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:53.268 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:53.268 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:53.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:53.289 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:44:53.289 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:44:53.289 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.23071 -> 3.23071 (expected baseline 3.13748) 2024-09-16T11:44:53.289 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:44:53.289 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 18 -> 18 pgs (1.6 -> 1.6 * mean) 2024-09-16T11:44:53.289 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:53.289 INFO:teuthology.orchestra.run.smithi070.stdout:oload 1000 2024-09-16T11:44:53.289 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 3 2024-09-16T11:44:53.289 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:44:53.289 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0033 2024-09-16T11:44:53.289 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0330 2024-09-16T11:44:53.290 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:44:53.301 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T11:44:53.302 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-16T11:44:53.343 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:53.341+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:53.390 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:53.390 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:53.399 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:53.443 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:53.441+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:53.471 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:44:53.515 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:53.513+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:53.519 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:53.519 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:53.529 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:53.616 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:53.614+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:44:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:44:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.23071 -> 3.23071 (expected baseline 3.13748) 2024-09-16T11:44:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:44:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 18 -> 18 pgs (1.6 -> 1.6 * mean) 2024-09-16T11:44:53.647 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:oload 110 2024-09-16T11:44:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 0.05 2024-09-16T11:44:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:44:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0033 2024-09-16T11:44:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0036 2024-09-16T11:44:53.648 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:44:53.660 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T11:44:53.660 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-16T11:44:53.669 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:53.669 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:53.678 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:53.744 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:53.742+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:53.776 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:53.776 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:53.784 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:53.818 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:53.816+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:53.883 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:53.883 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:53.891 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:53.919 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:53.917+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:53.961 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 90 (0%) 2024-09-16T11:44:53.961 INFO:teuthology.orchestra.run.smithi070.stdout:avg 11.25 2024-09-16T11:44:53.961 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 3.23071 -> 3.23071 (expected baseline 3.13748) 2024-09-16T11:44:53.961 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 7 -> 7 pgs (0.622222 -> 0.622222 * mean) 2024-09-16T11:44:53.961 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.1 with 18 -> 18 pgs (1.6 -> 1.6 * mean) 2024-09-16T11:44:53.961 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:44:53.961 INFO:teuthology.orchestra.run.smithi070.stdout:oload 110 2024-09-16T11:44:53.962 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 1 2024-09-16T11:44:53.962 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T11:44:53.962 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0033 2024-09-16T11:44:53.962 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0036 2024-09-16T11:44:53.962 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T11:44:53.989 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:53.989 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:53.998 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:54.041 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:54.039+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:54.095 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:54.096 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:54.104 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:54.145 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:54.143+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:54.222 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:54.220+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:54.242 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:54.243 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:54.251 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:54.322 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:54.320+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:54.349 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:54.349 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:54.358 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:54.424 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:54.422+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:54.454 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:54.454 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:54.462 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:54.484 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T11:44:54.525 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:54.523+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:54.570 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:54.571 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:54.580 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:54.664 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:54.662+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:54.719 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:54.719 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:54.727 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:54.728 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:54.725+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:54.828 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:54.826+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:54.865 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:54.865 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:54.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:54.929 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:54.927+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:54.974 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:54.974 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:54.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:55.030 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:55.028+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:55.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:44:55.122 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:55.122 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:55.131 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:55.132 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:55.129+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:55.194 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:55.210 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:55.225 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:55.225 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:55.232 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:55.230+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:55.240 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:55.240 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:55.256 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:55.256 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:55.271 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:55.271 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:55.272 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:55.272 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89607 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1327104~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:44:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.342937547999998, 2024-09-16T11:44:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.033363680999999999, 2024-09-16T11:44:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.273 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89607 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929100+0000", 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001870776 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929104+0000", 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1160000000000001e-06 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929118+0000", 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3891e-05 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929126+0000", 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6590000000000007e-06 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960508+0000", 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031381920000000001 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960548+0000", 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0734e-05 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960593+0000", 2024-09-16T11:44:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4585000000000001e-05 2024-09-16T11:44:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89611 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1687552~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:44:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.340955588, 2024-09-16T11:44:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031799929999999997, 2024-09-16T11:44:55.277 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89611 2024-09-16T11:44:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:44:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929212+0000", 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0800000000000001e-07 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929213+0000", 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6509999999999999e-06 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929218+0000", 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9899999999999997e-06 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929221+0000", 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1140000000000002e-06 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960926+0000", 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031704436000000003 2024-09-16T11:44:55.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960966+0000", 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0055999999999997e-05 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961011+0000", 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5374999999999998e-05 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89612 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1777664~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.340942379000001, 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031893013999999997, 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89612 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6e-07 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929226+0000", 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5099999999999999e-06 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929231+0000", 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4889999999999997e-06 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929234+0000", 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7410000000000001e-06 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961033+0000", 2024-09-16T11:44:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031799279 2024-09-16T11:44:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961071+0000", 2024-09-16T11:44:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7759000000000002e-05 2024-09-16T11:44:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961118+0000", 2024-09-16T11:44:55.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6975999999999999e-05 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89613 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1867776~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.340930239999999, 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031996359000000002, 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.286 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89613 2024-09-16T11:44:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:44:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8999999999999998e-07 2024-09-16T11:44:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929243+0000", 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8719999999999999e-06 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929249+0000", 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6229999999999999e-06 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929253+0000", 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961145+0000", 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031892648000000003 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961188+0000", 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2466999999999998e-05 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961233+0000", 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5049999999999997e-05 2024-09-16T11:44:55.290 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89615 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1957888~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.34091037, 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032076532999999997, 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.291 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89615 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:44:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929259+0000", 2024-09-16T11:44:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7269999999999999e-06 2024-09-16T11:44:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929264+0000", 2024-09-16T11:44:55.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.524e-06 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929267+0000", 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.994e-06 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961252+0000", 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031985188999999997 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961291+0000", 2024-09-16T11:44:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8263000000000003e-05 2024-09-16T11:44:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961333+0000", 2024-09-16T11:44:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2491999999999999e-05 2024-09-16T11:44:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89617 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 2048000~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.295 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.340896737000001, 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032185244000000002, 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89617 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:55.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0499999999999999e-07 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929272+0000", 2024-09-16T11:44:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.615e-06 2024-09-16T11:44:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929279+0000", 2024-09-16T11:44:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5459999999999997e-06 2024-09-16T11:44:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929282+0000", 2024-09-16T11:44:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.472e-06 2024-09-16T11:44:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961361+0000", 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032078441999999999 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961401+0000", 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0425e-05 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961455+0000", 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4438999999999998e-05 2024-09-16T11:44:55.299 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89637 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4235264~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:44:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.340876479999999, 2024-09-16T11:44:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032120463000000002, 2024-09-16T11:44:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.300 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89637 2024-09-16T11:44:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:44:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:44:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929291+0000", 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929292+0000", 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3120000000000001e-06 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929295+0000", 2024-09-16T11:44:55.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3349999999999998e-06 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929297+0000", 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0549999999999998e-06 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961346+0000", 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032048920000000002 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961379+0000", 2024-09-16T11:44:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2533000000000002e-05 2024-09-16T11:44:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961411+0000", 2024-09-16T11:44:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1921000000000002e-05 2024-09-16T11:44:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89639 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4325376~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.304 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:44:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.340867401000001, 2024-09-16T11:44:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032194749000000002, 2024-09-16T11:44:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89639 2024-09-16T11:44:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:44:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:44:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929300+0000", 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929301+0000", 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.018e-06 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929304+0000", 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8890000000000002e-06 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929306+0000", 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.883e-06 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961422+0000", 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032116899999999997 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961460+0000", 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7265999999999999e-05 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961494+0000", 2024-09-16T11:44:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4499000000000003e-05 2024-09-16T11:44:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89641 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4415488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:44:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.340859477999999, 2024-09-16T11:44:55.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032287719999999999, 2024-09-16T11:44:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89641 2024-09-16T11:44:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:44:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:44:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929308+0000", 2024-09-16T11:44:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1500000000000001e-07 2024-09-16T11:44:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929309+0000", 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.02e-06 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929312+0000", 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3000000000000002e-06 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929314+0000", 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0449999999999999e-06 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961521+0000", 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032206835000000003 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961561+0000", 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0580000000000001e-05 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961595+0000", 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3725e-05 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.313 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89643 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4505600~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.34085104, 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032377892999999998, 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89643 2024-09-16T11:44:55.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:55.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:44:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0100000000000001e-07 2024-09-16T11:44:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929317+0000", 2024-09-16T11:44:55.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.161e-06 2024-09-16T11:44:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929320+0000", 2024-09-16T11:44:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1580000000000001e-06 2024-09-16T11:44:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929322+0000", 2024-09-16T11:44:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9929999999999998e-06 2024-09-16T11:44:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961619+0000", 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032296392 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961659+0000", 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9805000000000003e-05 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961694+0000", 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5182999999999998e-05 2024-09-16T11:44:55.318 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89644 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4243456~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.340842661, 2024-09-16T11:44:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032467912000000002, 2024-09-16T11:44:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.319 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89644 2024-09-16T11:44:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:44:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0100000000000001e-07 2024-09-16T11:44:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929325+0000", 2024-09-16T11:44:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9899999999999999e-07 2024-09-16T11:44:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929328+0000", 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7539999999999998e-06 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929330+0000", 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3690000000000001e-06 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961725+0000", 2024-09-16T11:44:55.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032394088000000001 2024-09-16T11:44:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961759+0000", 2024-09-16T11:44:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4316999999999997e-05 2024-09-16T11:44:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961792+0000", 2024-09-16T11:44:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3284000000000002e-05 2024-09-16T11:44:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.323 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89645 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4595712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.340834589, 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032554313000000001, 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89645 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.324 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.99e-07 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929333+0000", 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9299999999999996e-07 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929336+0000", 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8459999999999999e-06 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929340+0000", 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6370000000000001e-06 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961814+0000", 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032474461000000003 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961852+0000", 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7877999999999999e-05 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961887+0000", 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4399e-05 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89646 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4333568~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.340824951999998, 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032641146000000003, 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.328 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89646 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.04e-07 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929343+0000", 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7000000000000003e-07 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929346+0000", 2024-09-16T11:44:55.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7350000000000001e-06 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929349+0000", 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7030000000000002e-06 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961916+0000", 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032567634999999998 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961950+0000", 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3510000000000003e-05 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961983+0000", 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3389000000000002e-05 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89647 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4685824~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:44:55.332 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.34081655, 2024-09-16T11:44:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032727342, 2024-09-16T11:44:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89647 2024-09-16T11:44:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:44:55.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931111+0000", 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001760462 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931114+0000", 2024-09-16T11:44:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.67e-06 2024-09-16T11:44:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931122+0000", 2024-09-16T11:44:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6059999999999997e-06 2024-09-16T11:44:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931126+0000", 2024-09-16T11:44:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-06 2024-09-16T11:44:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962005+0000", 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.030879125 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962044+0000", 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8779000000000003e-05 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962078+0000", 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4010000000000001e-05 2024-09-16T11:44:55.336 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89656 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4784128~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:44:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.338966501000002, 2024-09-16T11:44:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031742976999999999, 2024-09-16T11:44:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.337 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89656 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931201+0000", 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0699999999999999e-07 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931202+0000", 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5900000000000005e-07 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931204+0000", 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8439999999999998e-06 2024-09-16T11:44:55.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931206+0000", 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.871e-06 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962877+0000", 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031670831000000003 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962910+0000", 2024-09-16T11:44:55.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3405999999999998e-05 2024-09-16T11:44:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962943+0000", 2024-09-16T11:44:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2858999999999998e-05 2024-09-16T11:44:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89657 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5136384~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:55.341 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.338958689999998, 2024-09-16T11:44:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031827938, 2024-09-16T11:44:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89657 2024-09-16T11:44:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:55.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.28e-07 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931209+0000", 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9099999999999991e-07 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931213+0000", 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7639999999999999e-06 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931217+0000", 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4759999999999998e-06 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962964+0000", 2024-09-16T11:44:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031747598000000002 2024-09-16T11:44:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963003+0000", 2024-09-16T11:44:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8287000000000001e-05 2024-09-16T11:44:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963036+0000", 2024-09-16T11:44:55.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3593999999999997e-05 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89658 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4874240~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.338931195000001, 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031905067000000002, 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.346 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89658 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4700000000000001e-07 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931238+0000", 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931245+0000", 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.64e-06 2024-09-16T11:44:55.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931248+0000", 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5719999999999999e-06 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963066+0000", 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031817326999999999 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963107+0000", 2024-09-16T11:44:55.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1220000000000002e-05 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963141+0000", 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3896999999999997e-05 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89659 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4964352~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.338914892999998, 2024-09-16T11:44:55.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031984460999999999, 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89659 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:44:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931254+0000", 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.776e-06 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931262+0000", 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1680000000000004e-06 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931266+0000", 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.732e-06 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963165+0000", 2024-09-16T11:44:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031898693999999998 2024-09-16T11:44:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963202+0000", 2024-09-16T11:44:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7517e-05 2024-09-16T11:44:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963236+0000", 2024-09-16T11:44:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4230000000000003e-05 2024-09-16T11:44:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89660 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5226496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.338896872999999, 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032062368000000001, 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.355 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89660 2024-09-16T11:44:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.39e-07 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931272+0000", 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931279+0000", 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:44:55.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931282+0000", 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.721e-06 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963266+0000", 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031983971999999999 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963300+0000", 2024-09-16T11:44:55.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3318000000000002e-05 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963332+0000", 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2768000000000002e-05 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89661 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5054464~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.338880449000001, 2024-09-16T11:44:55.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032139394000000002, 2024-09-16T11:44:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89661 2024-09-16T11:44:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:44:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933104+0000", 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00181734 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933107+0000", 2024-09-16T11:44:55.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5140000000000002e-06 2024-09-16T11:44:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933115+0000", 2024-09-16T11:44:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:44:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933124+0000", 2024-09-16T11:44:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8910000000000001e-06 2024-09-16T11:44:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963354+0000", 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.030230825999999999 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963392+0000", 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7765000000000002e-05 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963426+0000", 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3603000000000003e-05 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.363 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.364 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.364 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:55.365 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:55.366 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:55.366 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:55.366 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:55.367 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:55.367 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:55.362+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:55.466 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:55.463+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:55.510 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:55.510 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:55.511 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:55.521 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:55.567 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:55.564+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:55.624 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:55.624 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:55.625 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:55.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:55.667 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:55.665+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:55.768 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:55.766+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:55.772 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:55.772 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:55.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:55.772 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:55.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.773 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19364 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1171456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.773 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:55.773 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.256525445999998, 2024-09-16T11:44:55.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049001805000000002, 2024-09-16T11:44:55.773 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.773 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.773 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.773 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.773 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.773 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19364 2024-09-16T11:44:55.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.773 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.774 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.774 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:55.774 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.774 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.774 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.774 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:55.774 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.774 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.774 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.774 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:44:55.774 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:44:55.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513280+0000", 2024-09-16T11:44:55.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7090000000000001e-06 2024-09-16T11:44:55.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513286+0000", 2024-09-16T11:44:55.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5840000000000002e-06 2024-09-16T11:44:55.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513290+0000", 2024-09-16T11:44:55.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3680000000000004e-06 2024-09-16T11:44:55.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562121+0000", 2024-09-16T11:44:55.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048831324000000002 2024-09-16T11:44:55.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562195+0000", 2024-09-16T11:44:55.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3415000000000001e-05 2024-09-16T11:44:55.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562280+0000", 2024-09-16T11:44:55.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4975999999999994e-05 2024-09-16T11:44:55.777 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.777 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.777 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.777 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19365 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1490944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.777 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:55.777 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.256505707000002, 2024-09-16T11:44:55.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049335349000000001, 2024-09-16T11:44:55.778 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.778 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.778 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.778 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.778 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.778 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19365 2024-09-16T11:44:55.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.778 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:55.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513300+0000", 2024-09-16T11:44:55.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3020000000000002e-06 2024-09-16T11:44:55.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513306+0000", 2024-09-16T11:44:55.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8780000000000003e-06 2024-09-16T11:44:55.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513310+0000", 2024-09-16T11:44:55.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1309999999999999e-06 2024-09-16T11:44:55.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562322+0000", 2024-09-16T11:44:55.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049011635999999997 2024-09-16T11:44:55.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562544+0000", 2024-09-16T11:44:55.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022168000000000001 2024-09-16T11:44:55.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562633+0000", 2024-09-16T11:44:55.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9350000000000003e-05 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19366 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1916928~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.256488775000001, 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049865073000000003, 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.782 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19366 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.783 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515102+0000", 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017877419999999999 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515106+0000", 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4450000000000001e-06 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515113+0000", 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7680000000000008e-06 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515117+0000", 2024-09-16T11:44:55.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:44:55.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562756+0000", 2024-09-16T11:44:55.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047638579 2024-09-16T11:44:55.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563093+0000", 2024-09-16T11:44:55.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000337516 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563180+0000", 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6420000000000003e-05 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19371 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3194880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.254591568000002, 2024-09-16T11:44:55.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048948356999999998, 2024-09-16T11:44:55.787 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.787 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.787 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.787 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.787 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.787 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19371 2024-09-16T11:44:55.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.787 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.787 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.787 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:55.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515214+0000", 2024-09-16T11:44:55.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5969999999999999e-06 2024-09-16T11:44:55.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.789 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.789 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515219+0000", 2024-09-16T11:44:55.789 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4559999999999999e-06 2024-09-16T11:44:55.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.789 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.789 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515223+0000", 2024-09-16T11:44:55.789 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:44:55.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563991+0000", 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048768911999999998 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564063+0000", 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2010999999999999e-05 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564160+0000", 2024-09-16T11:44:55.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6617e-05 2024-09-16T11:44:55.791 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.791 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.791 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.791 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19372 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3620864~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.791 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:55.791 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.254577351000002, 2024-09-16T11:44:55.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049130502, 2024-09-16T11:44:55.791 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.791 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.791 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.791 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.792 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.792 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19372 2024-09-16T11:44:55.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.792 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.792 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.792 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:55.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.792 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.792 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:55.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515228+0000", 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515233+0000", 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515237+0000", 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.27e-06 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564199+0000", 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048961787999999999 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564275+0000", 2024-09-16T11:44:55.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6001000000000006e-05 2024-09-16T11:44:55.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564356+0000", 2024-09-16T11:44:55.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1941999999999996e-05 2024-09-16T11:44:55.795 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.795 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.795 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.795 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19373 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3940352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.254562950999997, 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049289588000000002, 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19373 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.796 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515241+0000", 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.797 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515242+0000", 2024-09-16T11:44:55.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.528e-06 2024-09-16T11:44:55.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515248+0000", 2024-09-16T11:44:55.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:55.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515251+0000", 2024-09-16T11:44:55.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2480000000000001e-06 2024-09-16T11:44:55.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564388+0000", 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049137313000000002 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564455+0000", 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7049000000000001e-05 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564530+0000", 2024-09-16T11:44:55.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4626e-05 2024-09-16T11:44:55.800 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.800 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.800 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.800 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19374 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 4046848~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.800 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:55.800 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.254548548000002, 2024-09-16T11:44:55.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049457133, 2024-09-16T11:44:55.800 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.800 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.800 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.800 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.801 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.801 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19374 2024-09-16T11:44:55.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.801 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:55.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:55.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:44:55.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:55.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515257+0000", 2024-09-16T11:44:55.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5740000000000001e-06 2024-09-16T11:44:55.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515262+0000", 2024-09-16T11:44:55.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2399999999999998e-06 2024-09-16T11:44:55.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515265+0000", 2024-09-16T11:44:55.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3909999999999998e-06 2024-09-16T11:44:55.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564555+0000", 2024-09-16T11:44:55.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049290002999999999 2024-09-16T11:44:55.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564628+0000", 2024-09-16T11:44:55.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2559999999999996e-05 2024-09-16T11:44:55.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564712+0000", 2024-09-16T11:44:55.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4040000000000002e-05 2024-09-16T11:44:55.804 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.804 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.804 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.805 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19375 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3301376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.805 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:55.805 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.254534405000001, 2024-09-16T11:44:55.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049606631999999998, 2024-09-16T11:44:55.805 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.805 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.805 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.805 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.805 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.805 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19375 2024-09-16T11:44:55.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515271+0000", 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5230000000000001e-06 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515276+0000", 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4650000000000002e-06 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515279+0000", 2024-09-16T11:44:55.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0630000000000002e-06 2024-09-16T11:44:55.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564738+0000", 2024-09-16T11:44:55.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049459001000000002 2024-09-16T11:44:55.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564780+0000", 2024-09-16T11:44:55.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1693999999999998e-05 2024-09-16T11:44:55.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564876+0000", 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5571999999999999e-05 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19376 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3727360~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.254520493000001, 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049720590000000002, 2024-09-16T11:44:55.809 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.810 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.810 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19376 2024-09-16T11:44:55.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.810 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.810 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.810 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:55.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517103+0000", 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001820276 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517108+0000", 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5410000000000002e-06 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517120+0000", 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2092e-05 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517125+0000", 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6829999999999996e-06 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564870+0000", 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047745040000000002 2024-09-16T11:44:55.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564921+0000", 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0538999999999998e-05 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565003+0000", 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2418999999999998e-05 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.814 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19492 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 434176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.814 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:55.814 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.252591637000002, 2024-09-16T11:44:55.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048926948999999997, 2024-09-16T11:44:55.814 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.814 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.814 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.814 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.814 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.814 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19492 2024-09-16T11:44:55.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.814 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3099999999999999e-07 2024-09-16T11:44:55.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517213+0000", 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6599999999999994e-07 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517216+0000", 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.706e-06 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517218+0000", 2024-09-16T11:44:55.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.035e-06 2024-09-16T11:44:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565959+0000", 2024-09-16T11:44:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048741380000000001 2024-09-16T11:44:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566036+0000", 2024-09-16T11:44:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7362999999999998e-05 2024-09-16T11:44:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566139+0000", 2024-09-16T11:44:55.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102268 2024-09-16T11:44:55.818 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.818 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.818 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.818 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19493 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 860160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.818 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:55.818 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.252569966000003, 2024-09-16T11:44:55.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049086604999999998, 2024-09-16T11:44:55.818 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.818 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.819 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.819 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.819 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.819 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19493 2024-09-16T11:44:55.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.819 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:55.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:44:55.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517234+0000", 2024-09-16T11:44:55.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.37e-07 2024-09-16T11:44:55.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517236+0000", 2024-09-16T11:44:55.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7150000000000001e-06 2024-09-16T11:44:55.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517243+0000", 2024-09-16T11:44:55.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9489999999999999e-06 2024-09-16T11:44:55.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517246+0000", 2024-09-16T11:44:55.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.636e-06 2024-09-16T11:44:55.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566163+0000", 2024-09-16T11:44:55.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048917284999999998 2024-09-16T11:44:55.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566237+0000", 2024-09-16T11:44:55.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3196999999999997e-05 2024-09-16T11:44:55.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566320+0000", 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3286000000000002e-05 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19494 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1179648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.252549350000002, 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049259562999999999, 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.823 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.824 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.824 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.824 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19494 2024-09-16T11:44:55.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.824 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:55.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:55.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:44:55.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:44:55.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517256+0000", 2024-09-16T11:44:55.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.888e-06 2024-09-16T11:44:55.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517265+0000", 2024-09-16T11:44:55.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8629999999999997e-06 2024-09-16T11:44:55.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517269+0000", 2024-09-16T11:44:55.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2039999999999999e-06 2024-09-16T11:44:55.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566352+0000", 2024-09-16T11:44:55.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049083107000000001 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566429+0000", 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7025000000000002e-05 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566514+0000", 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4121000000000003e-05 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19495 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1286144~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.252529926999998, 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049492880000000003, 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19495 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517274+0000", 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:44:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517276+0000", 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8950000000000001e-06 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517281+0000", 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3879999999999999e-06 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517285+0000", 2024-09-16T11:44:55.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5930000000000002e-06 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566579+0000", 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049294175000000003 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566670+0000", 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1096999999999999e-05 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566766+0000", 2024-09-16T11:44:55.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6355999999999997e-05 2024-09-16T11:44:55.832 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.832 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.832 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.832 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19496 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1605632~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.832 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:55.832 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.252514841999997, 2024-09-16T11:44:55.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049632981999999999, 2024-09-16T11:44:55.832 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.832 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.832 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19496 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:44:55.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517289+0000", 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517290+0000", 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.581e-06 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.834 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517300+0000", 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517304+0000", 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566789+0000", 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049485102000000003 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566843+0000", 2024-09-16T11:44:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3937000000000002e-05 2024-09-16T11:44:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566921+0000", 2024-09-16T11:44:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8397000000000002e-05 2024-09-16T11:44:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19497 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 114688~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.252495146999998, 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049761367000000001, 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19497 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519116+0000", 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018079439999999999 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519120+0000", 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2760000000000002e-06 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519134+0000", 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3468999999999999e-05 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519140+0000", 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.516e-06 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.839 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566943+0000", 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047802561 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566982+0000", 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9269e-05 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567070+0000", 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7331999999999997e-05 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.840 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19504 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 966656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.250566659999997, 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049045863000000002, 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19504 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.841 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:55.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519239+0000", 2024-09-16T11:44:55.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.962e-06 2024-09-16T11:44:55.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519244+0000", 2024-09-16T11:44:55.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5480000000000001e-06 2024-09-16T11:44:55.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519248+0000", 2024-09-16T11:44:55.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.641e-06 2024-09-16T11:44:55.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568150+0000", 2024-09-16T11:44:55.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048901864000000003 2024-09-16T11:44:55.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568208+0000", 2024-09-16T11:44:55.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8399000000000001e-05 2024-09-16T11:44:55.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568283+0000", 2024-09-16T11:44:55.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4105999999999998e-05 2024-09-16T11:44:55.845 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.845 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.845 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.845 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19505 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1392640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.845 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:55.845 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.250551354999999, 2024-09-16T11:44:55.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049202304000000002, 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19505 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:55.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:44:55.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:44:55.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519254+0000", 2024-09-16T11:44:55.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.761e-06 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519257+0000", 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.343e-06 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519259+0000", 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0559999999999999e-06 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568307+0000", 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049047445000000002 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568382+0000", 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5574999999999994e-05 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568454+0000", 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1799999999999997e-05 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.849 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19506 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1712128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.250541317, 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049341532, 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19506 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.850 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:44:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521102+0000", 2024-09-16T11:44:55.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018404700000000001 2024-09-16T11:44:55.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521106+0000", 2024-09-16T11:44:55.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8820000000000003e-06 2024-09-16T11:44:55.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521119+0000", 2024-09-16T11:44:55.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2546e-05 2024-09-16T11:44:55.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521125+0000", 2024-09-16T11:44:55.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0249999999999999e-06 2024-09-16T11:44:55.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568484+0000", 2024-09-16T11:44:55.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047359120999999997 2024-09-16T11:44:55.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568542+0000", 2024-09-16T11:44:55.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8360999999999999e-05 2024-09-16T11:44:55.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568604+0000", 2024-09-16T11:44:55.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1126999999999999e-05 2024-09-16T11:44:55.854 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.854 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.854 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.854 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19515 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1499136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:55.854 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:55.854 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.248549032, 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048949431000000002, 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19515 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:44:55.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523130+0000", 2024-09-16T11:44:55.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00187604 2024-09-16T11:44:55.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523134+0000", 2024-09-16T11:44:55.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:44:55.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523144+0000", 2024-09-16T11:44:55.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2010000000000005e-06 2024-09-16T11:44:55.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523149+0000", 2024-09-16T11:44:55.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0039999999999999e-06 2024-09-16T11:44:55.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570113+0000", 2024-09-16T11:44:55.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.046964517999999997 2024-09-16T11:44:55.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570157+0000", 2024-09-16T11:44:55.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4056e-05 2024-09-16T11:44:55.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570204+0000", 2024-09-16T11:44:55.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6591999999999997e-05 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:44111 4.8 4:17f3f47e:::rbd_data.11c1e170f5f0.000000000000002f:head [delete] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 45.786568750999997, 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93903426400000001, 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:55.859 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:55.860 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 44111 2024-09-16T11:44:55.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.860 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:55.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:55.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:55.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:55.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:55.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:55.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:44:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7199999999999998e-07 2024-09-16T11:44:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983239+0000", 2024-09-16T11:44:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9879999999999998e-06 2024-09-16T11:44:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983248+0000", 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.371e-06 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983253+0000", 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1150000000000002e-06 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983303+0000", 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0380000000000002e-05 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983311+0000", 2024-09-16T11:44:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7519999999999996e-06 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920618+0000", 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93730672000000004 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920730+0000", 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011265900000000001 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920842+0000", 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111638 2024-09-16T11:44:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.921503+0000", 2024-09-16T11:44:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066074299999999999 2024-09-16T11:44:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922208+0000", 2024-09-16T11:44:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00070492899999999997 2024-09-16T11:44:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922233+0000", 2024-09-16T11:44:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5922999999999999e-05 2024-09-16T11:44:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922269+0000", 2024-09-16T11:44:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5373999999999997e-05 2024-09-16T11:44:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:55.865 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:55.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:55.908 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:55.906+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:55.971 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:55.968+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:56.005 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:56.005 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:56.013 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:56.097 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:56.095+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:56.149 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:56.150 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:56.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:56.199 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:56.197+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:56.273 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:56.271+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:56.297 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16853 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3375104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.947435042999999, 2024-09-16T11:44:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027791907000000001, 2024-09-16T11:44:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16853 2024-09-16T11:44:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:44:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347568+0000", 2024-09-16T11:44:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:44:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347570+0000", 2024-09-16T11:44:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:44:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347588+0000", 2024-09-16T11:44:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7914000000000002e-05 2024-09-16T11:44:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347592+0000", 2024-09-16T11:44:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3150000000000002e-06 2024-09-16T11:44:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375199+0000", 2024-09-16T11:44:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027606065999999999 2024-09-16T11:44:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375276+0000", 2024-09-16T11:44:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7527000000000005e-05 2024-09-16T11:44:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375359+0000", 2024-09-16T11:44:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3238000000000005e-05 2024-09-16T11:44:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16854 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.947405345999996, 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027945148999999999, 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16854 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347598+0000", 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347600+0000", 2024-09-16T11:44:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:44:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347606+0000", 2024-09-16T11:44:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0580000000000003e-06 2024-09-16T11:44:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347609+0000", 2024-09-16T11:44:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:44:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375386+0000", 2024-09-16T11:44:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027776809 2024-09-16T11:44:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375461+0000", 2024-09-16T11:44:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4868000000000002e-05 2024-09-16T11:44:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375542+0000", 2024-09-16T11:44:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1253000000000003e-05 2024-09-16T11:44:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16855 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 4120576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.947389006999998, 2024-09-16T11:44:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028111094999999999, 2024-09-16T11:44:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16855 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347614+0000", 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347616+0000", 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347623+0000", 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0099999999999998e-06 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347626+0000", 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2140000000000001e-06 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375569+0000", 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027942649999999999 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375643+0000", 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3949999999999995e-05 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375725+0000", 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1992999999999999e-05 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16856 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3801088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.947366936000002, 2024-09-16T11:44:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028268272000000001, 2024-09-16T11:44:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16856 2024-09-16T11:44:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347636+0000", 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347638+0000", 2024-09-16T11:44:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.86e-06 2024-09-16T11:44:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347644+0000", 2024-09-16T11:44:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1739999999999997e-06 2024-09-16T11:44:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347656+0000", 2024-09-16T11:44:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2198000000000001e-05 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375751+0000", 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028094507000000001 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375824+0000", 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3178999999999998e-05 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375904+0000", 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9985000000000001e-05 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16857 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.947340433999997, 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028275403000000001, 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16857 2024-09-16T11:44:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.46e-07 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347666+0000", 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8739999999999997e-06 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347672+0000", 2024-09-16T11:44:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1059999999999996e-06 2024-09-16T11:44:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347676+0000", 2024-09-16T11:44:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:44:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375930+0000", 2024-09-16T11:44:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028254042 2024-09-16T11:44:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375937+0000", 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5519999999999999e-06 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16858 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.947322788999998, 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028284592000000001, 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16858 2024-09-16T11:44:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347683+0000", 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6809999999999998e-06 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347689+0000", 2024-09-16T11:44:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:44:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347695+0000", 2024-09-16T11:44:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0020000000000003e-06 2024-09-16T11:44:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375958+0000", 2024-09-16T11:44:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028263870999999999 2024-09-16T11:44:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375964+0000", 2024-09-16T11:44:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8209999999999998e-06 2024-09-16T11:44:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16992 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.945748101, 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027704776, 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16992 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:44:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349255+0000", 2024-09-16T11:44:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:44:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349260+0000", 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5589999999999996e-06 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349267+0000", 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4019999999999999e-06 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349270+0000", 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6780000000000002e-06 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376953+0000", 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682933 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376959+0000", 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16993 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.945726776000001, 2024-09-16T11:44:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027707908999999999, 2024-09-16T11:44:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16993 2024-09-16T11:44:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349278+0000", 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.629e-06 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349291+0000", 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614e-05 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349295+0000", 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1189999999999999e-06 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376978+0000", 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682445 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376984+0000", 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7590000000000003e-06 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16994 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.945702765, 2024-09-16T11:44:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027708753999999999, 2024-09-16T11:44:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16994 2024-09-16T11:44:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349302+0000", 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.722e-06 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349308+0000", 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349312+0000", 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5310000000000002e-06 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377003+0000", 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027691308000000001 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377008+0000", 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4550000000000003e-06 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16999 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.945559887000002, 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027705270000000001, 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16999 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.46e-07 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349445+0000", 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.637e-06 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349451+0000", 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9959999999999999e-06 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349454+0000", 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.647e-06 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377142+0000", 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027687942 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377148+0000", 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7019999999999998e-06 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17000 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.945544380000001, 2024-09-16T11:44:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.02771881, 2024-09-16T11:44:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17000 2024-09-16T11:44:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0600000000000001e-07 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349460+0000", 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.736e-06 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349466+0000", 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2419999999999997e-06 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349470+0000", 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5480000000000002e-06 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377171+0000", 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027701282000000001 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377177+0000", 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5960000000000003e-06 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17001 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.945528629999998, 2024-09-16T11:44:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027739243, 2024-09-16T11:44:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:44:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17001 2024-09-16T11:44:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:44:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349476+0000", 2024-09-16T11:44:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T11:44:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349482+0000", 2024-09-16T11:44:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2389999999999999e-06 2024-09-16T11:44:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349486+0000", 2024-09-16T11:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5980000000000001e-06 2024-09-16T11:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377206+0000", 2024-09-16T11:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027720037999999999 2024-09-16T11:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377213+0000", 2024-09-16T11:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3440000000000003e-06 2024-09-16T11:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17433 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 5898240~4888] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.909564975999999, 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027709398, 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17433 2024-09-16T11:44:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:44:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385438+0000", 2024-09-16T11:44:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385439+0000", 2024-09-16T11:44:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5120000000000001e-06 2024-09-16T11:44:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385445+0000", 2024-09-16T11:44:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4e-06 2024-09-16T11:44:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385448+0000", 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1669999999999999e-06 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412916+0000", 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027468137 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412988+0000", 2024-09-16T11:44:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1606000000000005e-05 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413147+0000", 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015924299999999999 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17434 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6004736~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.909550385999999, 2024-09-16T11:44:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027777461, 2024-09-16T11:44:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17434 2024-09-16T11:44:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:44:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:44:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385454+0000", 2024-09-16T11:44:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6729999999999999e-06 2024-09-16T11:44:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385459+0000", 2024-09-16T11:44:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4079999999999997e-06 2024-09-16T11:44:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385463+0000", 2024-09-16T11:44:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.269e-06 2024-09-16T11:44:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413157+0000", 2024-09-16T11:44:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027694611000000001 2024-09-16T11:44:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413193+0000", 2024-09-16T11:44:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5673999999999998e-05 2024-09-16T11:44:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413230+0000", 2024-09-16T11:44:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6454e-05 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17435 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6324224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.909536232999997, 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929059999999999, 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17435 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385467+0000", 2024-09-16T11:44:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385468+0000", 2024-09-16T11:44:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6300000000000001e-06 2024-09-16T11:44:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385474+0000", 2024-09-16T11:44:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1299999999999998e-06 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385478+0000", 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2880000000000001e-06 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413257+0000", 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027779608000000001 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413331+0000", 2024-09-16T11:44:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3639000000000004e-05 2024-09-16T11:44:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413395+0000", 2024-09-16T11:44:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4424999999999994e-05 2024-09-16T11:44:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17436 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7176192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.909520571000002, 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028016718999999999, 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17436 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385484+0000", 2024-09-16T11:44:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5179999999999999e-06 2024-09-16T11:44:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385489+0000", 2024-09-16T11:44:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7049999999999996e-06 2024-09-16T11:44:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385493+0000", 2024-09-16T11:44:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T11:44:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413422+0000", 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929611999999999 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413462+0000", 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9585000000000001e-05 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413499+0000", 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6644999999999999e-05 2024-09-16T11:44:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17437 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6750208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.909506155999999, 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028167168999999999, 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17437 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385497+0000", 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385498+0000", 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.592e-06 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385504+0000", 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4090000000000002e-06 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385513+0000", 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4460000000000007e-06 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413526+0000", 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028012866000000001 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413601+0000", 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5388000000000004e-05 2024-09-16T11:44:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413663+0000", 2024-09-16T11:44:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.211e-05 2024-09-16T11:44:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17438 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7495680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.909483397000002, 2024-09-16T11:44:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028224439, 2024-09-16T11:44:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17438 2024-09-16T11:44:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4900000000000001e-07 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385521+0000", 2024-09-16T11:44:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:44:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385526+0000", 2024-09-16T11:44:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2680000000000002e-06 2024-09-16T11:44:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385529+0000", 2024-09-16T11:44:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2040000000000002e-06 2024-09-16T11:44:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413674+0000", 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028144226000000001 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413710+0000", 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6665000000000002e-05 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413743+0000", 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3155999999999999e-05 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17439 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7921664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.909422243000002, 2024-09-16T11:44:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028326131000000001, 2024-09-16T11:44:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17439 2024-09-16T11:44:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:44:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385581+0000", 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9299999999999998e-07 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385583+0000", 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6220000000000001e-06 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385592+0000", 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5220000000000007e-06 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385595+0000", 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8669999999999998e-06 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413771+0000", 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028175875999999999 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413845+0000", 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4382999999999996e-05 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413906+0000", 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1367999999999999e-05 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17440 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 60.909405034999999, 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028391117, 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17440 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:44:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385598+0000", 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385603+0000", 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385607+0000", 2024-09-16T11:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4869999999999998e-06 2024-09-16T11:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385609+0000", 2024-09-16T11:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2759999999999999e-06 2024-09-16T11:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413916+0000", 2024-09-16T11:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028307253000000001 2024-09-16T11:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413955+0000", 2024-09-16T11:44:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8652000000000002e-05 2024-09-16T11:44:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413989+0000", 2024-09-16T11:44:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3587999999999998e-05 2024-09-16T11:44:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.385 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:56.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:56.386 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:56.384+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:56.487 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:56.485+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:56.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:44:56.540 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:56.540 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:56.540 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:44:56.540 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:56.550 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:56.591 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:56.589+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:56.689 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:56.687+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:56.690 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:56.690 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:56.690 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:56.690 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:56.690 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:56.699 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:56.789 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:56.787+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:56.839 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:56.839 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:44:56.839 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:44:56.839 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:44:56.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.839 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19545 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4415488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.839 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:56.840 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.312661024000001, 2024-09-16T11:44:56.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043429581000000002, 2024-09-16T11:44:56.840 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.840 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.840 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.840 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.840 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.840 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19545 2024-09-16T11:44:56.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.840 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523839+0000", 2024-09-16T11:44:56.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0220000000000004e-06 2024-09-16T11:44:56.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523845+0000", 2024-09-16T11:44:56.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6500000000000001e-06 2024-09-16T11:44:56.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523848+0000", 2024-09-16T11:44:56.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1879999999999998e-06 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567092+0000", 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043244145999999997 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567176+0000", 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3825999999999993e-05 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567263+0000", 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7423999999999995e-05 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19546 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4734976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.312643184000002, 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043599231000000002, 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.844 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19546 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523954+0000", 2024-09-16T11:44:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010217099999999999 2024-09-16T11:44:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523957+0000", 2024-09-16T11:44:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3270000000000001e-06 2024-09-16T11:44:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523966+0000", 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6489999999999997e-06 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523971+0000", 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567291+0000", 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043319931999999998 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567360+0000", 2024-09-16T11:44:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8628999999999996e-05 2024-09-16T11:44:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567451+0000", 2024-09-16T11:44:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1012000000000002e-05 2024-09-16T11:44:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.848 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19547 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4841472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.312518326999999, 2024-09-16T11:44:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043658858000000002, 2024-09-16T11:44:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19547 2024-09-16T11:44:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525102+0000", 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011257730000000001 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525106+0000", 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9910000000000004e-06 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525116+0000", 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0231000000000001e-05 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525137+0000", 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0828000000000001e-05 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567478+0000", 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.042340467999999999 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567554+0000", 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6074000000000004e-05 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567635+0000", 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1493000000000001e-05 2024-09-16T11:44:56.852 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19553 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4947968~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.311291265000001, 2024-09-16T11:44:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043450397000000002, 2024-09-16T11:44:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.853 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19553 2024-09-16T11:44:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1799999999999999e-07 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525205+0000", 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.062e-06 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525208+0000", 2024-09-16T11:44:56.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8739999999999999e-06 2024-09-16T11:44:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525209+0000", 2024-09-16T11:44:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.725e-06 2024-09-16T11:44:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568520+0000", 2024-09-16T11:44:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043310226 2024-09-16T11:44:56.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568569+0000", 2024-09-16T11:44:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8959999999999999e-05 2024-09-16T11:44:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568654+0000", 2024-09-16T11:44:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5332000000000002e-05 2024-09-16T11:44:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19554 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5267456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.311282273000003, 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043659587, 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19554 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:44:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527103+0000", 2024-09-16T11:44:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018907419999999999 2024-09-16T11:44:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527108+0000", 2024-09-16T11:44:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6040000000000003e-06 2024-09-16T11:44:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527118+0000", 2024-09-16T11:44:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:44:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527122+0000", 2024-09-16T11:44:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9559999999999996e-06 2024-09-16T11:44:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568688+0000", 2024-09-16T11:44:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041565116999999999 2024-09-16T11:44:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568768+0000", 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0474000000000002e-05 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568872+0000", 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104028 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.862 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19565 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6332416~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.309203701000001, 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043494890000000001, 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19565 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:56.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:44:56.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527292+0000", 2024-09-16T11:44:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.27e-07 2024-09-16T11:44:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527293+0000", 2024-09-16T11:44:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.917e-06 2024-09-16T11:44:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527299+0000", 2024-09-16T11:44:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6699999999999999e-06 2024-09-16T11:44:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527304+0000", 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6160000000000004e-06 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570683+0000", 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043379171000000001 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570733+0000", 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0485000000000002e-05 2024-09-16T11:44:56.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570786+0000", 2024-09-16T11:44:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2604e-05 2024-09-16T11:44:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19566 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6758400~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.309187367, 2024-09-16T11:44:56.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043643183000000002, 2024-09-16T11:44:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19566 2024-09-16T11:44:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:56.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:44:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527308+0000", 2024-09-16T11:44:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:44:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527310+0000", 2024-09-16T11:44:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.68e-06 2024-09-16T11:44:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527316+0000", 2024-09-16T11:44:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9730000000000002e-06 2024-09-16T11:44:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527319+0000", 2024-09-16T11:44:56.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.027e-06 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570810+0000", 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043491525000000003 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570882+0000", 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.871 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570951+0000", 2024-09-16T11:44:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8782000000000007e-05 2024-09-16T11:44:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19567 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.309172660999998, 2024-09-16T11:44:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043786291999999997, 2024-09-16T11:44:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.872 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19567 2024-09-16T11:44:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:44:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529108+0000", 2024-09-16T11:44:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001785656 2024-09-16T11:44:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529113+0000", 2024-09-16T11:44:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1080000000000001e-06 2024-09-16T11:44:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529122+0000", 2024-09-16T11:44:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0550000000000005e-06 2024-09-16T11:44:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529127+0000", 2024-09-16T11:44:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9849999999999997e-06 2024-09-16T11:44:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570997+0000", 2024-09-16T11:44:56.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041869811 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571063+0000", 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6340999999999994e-05 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571108+0000", 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5336000000000001e-05 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19580 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 8036352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.307197360000004, 2024-09-16T11:44:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043451596000000002, 2024-09-16T11:44:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19580 2024-09-16T11:44:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.877 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:44:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531106+0000", 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001808179 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531110+0000", 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5990000000000004e-06 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531123+0000", 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2687000000000001e-05 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531129+0000", 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572642+0000", 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041513474000000002 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572691+0000", 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8989000000000001e-05 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572749+0000", 2024-09-16T11:44:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7806999999999997e-05 2024-09-16T11:44:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19709 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4849664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.305137319000004, 2024-09-16T11:44:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043537291999999998, 2024-09-16T11:44:56.881 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19709 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.882 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:44:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531358+0000", 2024-09-16T11:44:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:44:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531360+0000", 2024-09-16T11:44:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:44:56.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531365+0000", 2024-09-16T11:44:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.716e-06 2024-09-16T11:44:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531369+0000", 2024-09-16T11:44:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5410000000000001e-06 2024-09-16T11:44:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574763+0000", 2024-09-16T11:44:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394359 2024-09-16T11:44:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574831+0000", 2024-09-16T11:44:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7997999999999995e-05 2024-09-16T11:44:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574895+0000", 2024-09-16T11:44:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3427000000000001e-05 2024-09-16T11:44:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.885 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19710 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4956160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.305122040000001, 2024-09-16T11:44:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043674198999999997, 2024-09-16T11:44:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.886 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19710 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533105+0000", 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001731961 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533109+0000", 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3080000000000001e-06 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533120+0000", 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0625000000000001e-05 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533125+0000", 2024-09-16T11:44:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:44:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574923+0000", 2024-09-16T11:44:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041797187 2024-09-16T11:44:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.889 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574991+0000", 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8752999999999997e-05 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.575047+0000", 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5596000000000001e-05 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19719 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5488640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:56.890 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.303150291000001, 2024-09-16T11:44:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043412013999999999, 2024-09-16T11:44:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19719 2024-09-16T11:44:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:56.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:44:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533345+0000", 2024-09-16T11:44:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:44:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533347+0000", 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8139999999999999e-06 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533352+0000", 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4970000000000001e-06 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533355+0000", 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.711e-06 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576629+0000", 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043274295999999997 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576706+0000", 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6711000000000005e-05 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576757+0000", 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0509e-05 2024-09-16T11:44:56.894 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19720 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5808128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.303137642000003, 2024-09-16T11:44:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043577006000000001, 2024-09-16T11:44:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.895 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19720 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:44:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.35e-07 2024-09-16T11:44:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533359+0000", 2024-09-16T11:44:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:44:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533364+0000", 2024-09-16T11:44:56.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4349999999999996e-06 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533367+0000", 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2349999999999999e-06 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576786+0000", 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043418885999999997 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576862+0000", 2024-09-16T11:44:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6172000000000002e-05 2024-09-16T11:44:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576934+0000", 2024-09-16T11:44:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1891999999999995e-05 2024-09-16T11:44:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19721 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6234112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.899 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.303124122, 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043681656999999999, 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19721 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:56.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-07 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533373+0000", 2024-09-16T11:44:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.973e-06 2024-09-16T11:44:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533377+0000", 2024-09-16T11:44:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.403e-06 2024-09-16T11:44:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533379+0000", 2024-09-16T11:44:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.058e-06 2024-09-16T11:44:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576959+0000", 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043579433000000001 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577015+0000", 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5859000000000002e-05 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577052+0000", 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7620999999999998e-05 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.903 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19722 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6340608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.303113394999997, 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043854922999999997, 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19722 2024-09-16T11:44:56.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:44:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535103+0000", 2024-09-16T11:44:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017217020000000001 2024-09-16T11:44:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535107+0000", 2024-09-16T11:44:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2819999999999998e-06 2024-09-16T11:44:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535118+0000", 2024-09-16T11:44:56.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0148e-05 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535123+0000", 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577091+0000", 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041967930000000001 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577163+0000", 2024-09-16T11:44:56.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1853999999999993e-05 2024-09-16T11:44:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577236+0000", 2024-09-16T11:44:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3699e-05 2024-09-16T11:44:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19732 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6447104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:56.908 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.301253539000001, 2024-09-16T11:44:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043399048000000003, 2024-09-16T11:44:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19732 2024-09-16T11:44:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:56.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535242+0000", 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535243+0000", 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.793e-06 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535249+0000", 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.643e-06 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535252+0000", 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578501+0000", 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043248343000000002 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578576+0000", 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5544999999999996e-05 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578640+0000", 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4018999999999998e-05 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.912 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19733 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6873088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.301232698, 2024-09-16T11:44:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043549681, 2024-09-16T11:44:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.913 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19733 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535263+0000", 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535265+0000", 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9369999999999998e-06 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535271+0000", 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9719999999999997e-06 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535274+0000", 2024-09-16T11:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578668+0000", 2024-09-16T11:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043393355000000002 2024-09-16T11:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578744+0000", 2024-09-16T11:44:56.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6853000000000004e-05 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578812+0000", 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.737e-05 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19734 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7299072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.301216951000001, 2024-09-16T11:44:56.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043699189999999999, 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19734 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537101+0000", 2024-09-16T11:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001823098 2024-09-16T11:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537105+0000", 2024-09-16T11:44:56.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:44:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537119+0000", 2024-09-16T11:44:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614999999999999e-05 2024-09-16T11:44:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537125+0000", 2024-09-16T11:44:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:44:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578838+0000", 2024-09-16T11:44:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041712605999999999 2024-09-16T11:44:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578912+0000", 2024-09-16T11:44:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4396999999999999e-05 2024-09-16T11:44:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578977+0000", 2024-09-16T11:44:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4858999999999997e-05 2024-09-16T11:44:56.921 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19851 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4431872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.299136206999997, 2024-09-16T11:44:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043564540999999998, 2024-09-16T11:44:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:44:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.922 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19851 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539100+0000", 2024-09-16T11:44:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017412079999999999 2024-09-16T11:44:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539105+0000", 2024-09-16T11:44:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T11:44:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539116+0000", 2024-09-16T11:44:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1824999999999999e-05 2024-09-16T11:44:56.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539122+0000", 2024-09-16T11:44:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9859999999999999e-06 2024-09-16T11:44:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580764+0000", 2024-09-16T11:44:56.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041641299 2024-09-16T11:44:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580839+0000", 2024-09-16T11:44:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5167999999999996e-05 2024-09-16T11:44:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580923+0000", 2024-09-16T11:44:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4314999999999995e-05 2024-09-16T11:44:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.926 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55904 4.14 4:2d2dfe38:::rbd_data.11c1b01b799a.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2068993~28159] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T11:44:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.744759306999999, 2024-09-16T11:44:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.90010917700000004, 2024-09-16T11:44:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:44:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:44:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:44:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:44:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55904 2024-09-16T11:44:56.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:44:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:44:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:44:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:44:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:44:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:44:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091737+0000", 2024-09-16T11:44:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.254e-06 2024-09-16T11:44:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:44:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091759+0000", 2024-09-16T11:44:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2487999999999999e-05 2024-09-16T11:44:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:44:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091769+0000", 2024-09-16T11:44:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0199e-05 2024-09-16T11:44:56.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091774+0000", 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091830+0000", 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6700999999999998e-05 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091840+0000", 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0128e-05 2024-09-16T11:44:56.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:44:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990355+0000", 2024-09-16T11:44:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.89851413099999999 2024-09-16T11:44:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:44:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990509+0000", 2024-09-16T11:44:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000154442 2024-09-16T11:44:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:44:56.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990626+0000", 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000116928 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991526+0000", 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00089961800000000003 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991793+0000", 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026779000000000001 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:44:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991824+0000", 2024-09-16T11:44:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0380999999999999e-05 2024-09-16T11:44:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:44:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:44:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:44:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991845+0000", 2024-09-16T11:44:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0919000000000001e-05 2024-09-16T11:44:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:44:56.933 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:44:56.933 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:56.935 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:44:56.935 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:56.933+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:57.036 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:57.034+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:57.046 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:57.046 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:57.046 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:44:57.046 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:57.055 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:44:57.139 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:57.137+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:57.195 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:57.195 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:44:57.195 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:44:57.195 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:44:57.196 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:57.203 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:44:57.238 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:57.236+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:57.339 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:57.338+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:57.344 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:44:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:44:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980366341, 2024-09-16T11:44:57.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:44:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:44:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:44:57.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:44:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:44:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:44:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:44:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:44:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:44:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:44:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:44:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:44:57.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:44:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:57.350 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980344312, 2024-09-16T11:44:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:44:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:44:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:44:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:44:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:44:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:44:57.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:44:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:44:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:44:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:44:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:44:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:44:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:44:57.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:44:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:44:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:44:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.358 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980329546999997, 2024-09-16T11:44:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:44:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.359 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:44:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:44:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:44:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:44:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:44:57.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:44:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:44:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:44:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:44:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:44:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:44:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:44:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:44:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:44:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:44:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.366 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980309478999999, 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:44:57.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:44:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:44:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:44:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:44:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:44:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.370 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980278642999998, 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.371 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:44:57.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:44:57.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:44:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:44:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:44:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:44:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:44:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:44:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:44:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980259123000003, 2024-09-16T11:44:57.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:44:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:44:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:44:57.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:44:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:44:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:44:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:44:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:44:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:44:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:44:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:44:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:44:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:44:57.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:44:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:57.380 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980231543999999, 2024-09-16T11:44:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:44:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:44:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:57.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:44:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:44:57.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:44:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:44:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:44:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:44:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:44:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:44:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:44:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:44:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:44:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:44:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:44:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:44:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.385 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980212848999997, 2024-09-16T11:44:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:44:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.386 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:44:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:44:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:44:57.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:44:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:44:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:44:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:44:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:44:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:44:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:44:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:44:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:44:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:44:57.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:44:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:44:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:44:57.394 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.395 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980197840000002, 2024-09-16T11:44:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:44:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.396 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:44:57.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:44:57.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:44:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:44:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:44:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:44:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:44:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:44:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:44:57.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:44:57.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:44:57.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:44:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:44:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:44:57.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:44:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:44:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980165741999997, 2024-09-16T11:44:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:44:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.406 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:44:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.407 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:44:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:44:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:44:57.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:44:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:44:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:44:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:44:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:44:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:44:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:44:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:44:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:44:57.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:44:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:44:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:44:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.414 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980142123, 2024-09-16T11:44:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:44:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:44:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:44:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:44:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:44:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:44:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:44:57.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:44:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:44:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980127394, 2024-09-16T11:44:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:44:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:44:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:44:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:44:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:44:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:44:57.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:44:57.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980105569999999, 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:44:57.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:44:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:44:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:44:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:44:57.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:44:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:44:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:44:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:44:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.428 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980090920000002, 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:44:57.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:44:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:44:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:44:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:44:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:44:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.433 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980069346999997, 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:44:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:44:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:44:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:44:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:44:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:44:57.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:44:57.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980049901000001, 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:44:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:44:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:44:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:44:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:44:57.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:44:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:44:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:44:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:44:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.442 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.980013204000002, 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:57.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.447 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.447 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:57.447 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.979991257999998, 2024-09-16T11:44:57.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:44:57.447 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.447 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.447 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:44:57.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.979969371999999, 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.451 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.455 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.979954653, 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.459 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.460 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.460 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:57.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:44:57.462 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:44:57.462 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:57.460+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:57.496 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T11:44:57.563 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:57.561+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:57.572 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:44:57.572 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:44:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "pg_update_log_missing(4.1 epoch 105/28 rep_tid 4113 entries 105'338 (0'0) error 4:8493fd73:::rbd_header.11c1cab081b3:head by client.4545.0:93418 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 105'336)", 2024-09-16T11:44:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:57.570950+0000", 2024-09-16T11:44:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.00036049299999999998, 2024-09-16T11:44:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038593200000000002, 2024-09-16T11:44:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:44:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-16T11:44:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:44:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:44:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:57.570950+0000", 2024-09-16T11:44:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:44:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:57.570950+0000", 2024-09-16T11:44:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:44:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:44:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:57.570950+0000", 2024-09-16T11:44:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6499999999999999e-07 2024-09-16T11:44:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:57.570957+0000", 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9269999999999999e-06 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:57.570970+0000", 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2537999999999999e-05 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:57.570975+0000", 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.339e-06 2024-09-16T11:44:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:44:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:44:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:44:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:57.571016+0000", 2024-09-16T11:44:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0754000000000003e-05 2024-09-16T11:44:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:44:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:44:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-09-16T11:44:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-09-16T11:44:57.576 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:44:57.582 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:44:57.593 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:57.594 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:57.609 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:57.610 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:57.624 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:57.624 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:57.640 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:57.640 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:57.654 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:57.654 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:57.663 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:57.661+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:57.669 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:57.669 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:57.684 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:57.684 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:57.700 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:57.700 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:57.721 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:57.721 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:57.730 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:44:57.765 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:57.763+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:57.827 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:57.827 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:57.835 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:44:57.879 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:57.877+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:57.932 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:57.933 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:57.941 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:44:57.981 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:57.979+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:58.068 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:58.065+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:58.078 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:58.079 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:58.087 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:44:58.169 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:58.167+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:58.225 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:58.226 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:58.234 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:44:58.278 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:58.276+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:58.333 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:58.333 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:58.342 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:44:58.382 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:58.380+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:58.472 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:58.470+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:58.479 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:58.479 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:58.488 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:44:58.572 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:58.570+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:58.594 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:58.594 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:58.603 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:44:58.674 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:58.672+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:58.705 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:58.705 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:58.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:44:58.775 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:58.773+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:58.811 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:58.811 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:58.820 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:44:58.876 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:58.874+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:58.917 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:58.917 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:58.926 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:44:58.974 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:44:58.974 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:44:58.975 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T11:44:58.975 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.475605 2024-09-16T11:44:58.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0.47560503956113087 2024-09-16T11:44:58.976 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:44:58.974+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:44:59.025 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:59.026 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:59.035 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:44:59.079 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:44:59.077+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:44:59.136 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:59.136 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:59.145 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:44:59.179 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:59.176+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:59.280 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:59.278+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:44:59.283 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:59.283 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:59.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:44:59.380 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:44:59.378+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:44:59.390 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:59.390 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:59.398 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:44:59.483 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:44:59.481+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:44:59.505 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.4 primary-affinity to 0.475605 (8311692) 2024-09-16T11:44:59.512 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:44:59.543 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:59.544 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:59.554 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:44:59.595 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:59.593+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:59.683 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:44:59.681+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:44:59.695 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:59.695 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:59.704 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:44:59.711 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:44:59.784 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:44:59.781+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:44:59.805 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:44:59.805 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:44:59.814 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:44:59.864 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:44:59.864 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:44:59.880 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:44:59.880 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:44:59.884 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:44:59.882+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:44:59.895 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:44:59.895 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:44:59.910 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:44:59.910 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:44:59.914 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:44:59.914 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:44:59.914 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:44:59.914 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:44:59.914 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:44:59.923 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:44:59.926 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:44:59.926 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:44:59.949 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:44:59.949 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:44:59.965 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:44:59.965 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:44:59.980 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:44:59.981 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:44:59.985 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:44:59.983+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:00.027 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:00.027 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:45:00.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:45:00.028 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:45:00.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.028 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19545 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4415488~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.028 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:45:00.028 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.500780586000005, 2024-09-16T11:45:00.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043429581000000002, 2024-09-16T11:45:00.028 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.028 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.028 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.028 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.028 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.028 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19545 2024-09-16T11:45:00.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523834+0000", 2024-09-16T11:45:00.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:45:00.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523839+0000", 2024-09-16T11:45:00.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0220000000000004e-06 2024-09-16T11:45:00.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523845+0000", 2024-09-16T11:45:00.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6500000000000001e-06 2024-09-16T11:45:00.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523848+0000", 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1879999999999998e-06 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567092+0000", 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043244145999999997 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567176+0000", 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3825999999999993e-05 2024-09-16T11:45:00.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567263+0000", 2024-09-16T11:45:00.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7423999999999995e-05 2024-09-16T11:45:00.032 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.032 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.032 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.032 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19546 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4734976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.032 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:45:00.032 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.500762746000007, 2024-09-16T11:45:00.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043599231000000002, 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19546 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523852+0000", 2024-09-16T11:45:00.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523954+0000", 2024-09-16T11:45:00.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010217099999999999 2024-09-16T11:45:00.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523957+0000", 2024-09-16T11:45:00.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3270000000000001e-06 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523966+0000", 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6489999999999997e-06 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523971+0000", 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567291+0000", 2024-09-16T11:45:00.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043319931999999998 2024-09-16T11:45:00.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567360+0000", 2024-09-16T11:45:00.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8628999999999996e-05 2024-09-16T11:45:00.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567451+0000", 2024-09-16T11:45:00.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1012000000000002e-05 2024-09-16T11:45:00.036 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.036 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19547 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4841472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.500637889000004, 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043658858000000002, 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19547 2024-09-16T11:45:00.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523976+0000", 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525102+0000", 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011257730000000001 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525106+0000", 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9910000000000004e-06 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525116+0000", 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0231000000000001e-05 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525137+0000", 2024-09-16T11:45:00.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0828000000000001e-05 2024-09-16T11:45:00.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567478+0000", 2024-09-16T11:45:00.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.042340467999999999 2024-09-16T11:45:00.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567554+0000", 2024-09-16T11:45:00.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6074000000000004e-05 2024-09-16T11:45:00.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567635+0000", 2024-09-16T11:45:00.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1493000000000001e-05 2024-09-16T11:45:00.041 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.041 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.041 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.041 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19553 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4947968~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.041 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:45:00.041 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.499410827000005, 2024-09-16T11:45:00.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043450397000000002, 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19553 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525204+0000", 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1799999999999999e-07 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525205+0000", 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.062e-06 2024-09-16T11:45:00.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525208+0000", 2024-09-16T11:45:00.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8739999999999999e-06 2024-09-16T11:45:00.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525209+0000", 2024-09-16T11:45:00.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.725e-06 2024-09-16T11:45:00.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568520+0000", 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043310226 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568569+0000", 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8959999999999999e-05 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568654+0000", 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5332000000000002e-05 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.045 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.046 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19554 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5267456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.046 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:45:00.046 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.499401835, 2024-09-16T11:45:00.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043659587, 2024-09-16T11:45:00.046 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.046 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.046 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.046 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.046 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.046 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19554 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.525213+0000", 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527103+0000", 2024-09-16T11:45:00.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018907419999999999 2024-09-16T11:45:00.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527108+0000", 2024-09-16T11:45:00.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6040000000000003e-06 2024-09-16T11:45:00.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527118+0000", 2024-09-16T11:45:00.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:45:00.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.049 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527122+0000", 2024-09-16T11:45:00.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9559999999999996e-06 2024-09-16T11:45:00.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.049 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568688+0000", 2024-09-16T11:45:00.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041565116999999999 2024-09-16T11:45:00.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.049 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568768+0000", 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0474000000000002e-05 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568872+0000", 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104028 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19565 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6332416~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:45:00.050 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.497323262999998, 2024-09-16T11:45:00.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043494890000000001, 2024-09-16T11:45:00.051 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.051 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.051 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.051 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.051 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.051 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19565 2024-09-16T11:45:00.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.051 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:45:00.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527291+0000", 2024-09-16T11:45:00.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527292+0000", 2024-09-16T11:45:00.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.27e-07 2024-09-16T11:45:00.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527293+0000", 2024-09-16T11:45:00.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.917e-06 2024-09-16T11:45:00.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527299+0000", 2024-09-16T11:45:00.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6699999999999999e-06 2024-09-16T11:45:00.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527304+0000", 2024-09-16T11:45:00.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6160000000000004e-06 2024-09-16T11:45:00.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570683+0000", 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043379171000000001 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570733+0000", 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0485000000000002e-05 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570786+0000", 2024-09-16T11:45:00.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2604e-05 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19566 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6758400~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.497306929000004, 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043643183000000002, 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.055 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.056 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19566 2024-09-16T11:45:00.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.056 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:45:00.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527307+0000", 2024-09-16T11:45:00.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527308+0000", 2024-09-16T11:45:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:45:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527310+0000", 2024-09-16T11:45:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.68e-06 2024-09-16T11:45:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527316+0000", 2024-09-16T11:45:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9730000000000002e-06 2024-09-16T11:45:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527319+0000", 2024-09-16T11:45:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.027e-06 2024-09-16T11:45:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570810+0000", 2024-09-16T11:45:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043491525000000003 2024-09-16T11:45:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570882+0000", 2024-09-16T11:45:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:45:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570951+0000", 2024-09-16T11:45:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8782000000000007e-05 2024-09-16T11:45:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19567 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:45:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.497292223000002, 2024-09-16T11:45:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043786291999999997, 2024-09-16T11:45:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19567 2024-09-16T11:45:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:45:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.527322+0000", 2024-09-16T11:45:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529108+0000", 2024-09-16T11:45:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001785656 2024-09-16T11:45:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529113+0000", 2024-09-16T11:45:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1080000000000001e-06 2024-09-16T11:45:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529122+0000", 2024-09-16T11:45:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0550000000000005e-06 2024-09-16T11:45:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529127+0000", 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9849999999999997e-06 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570997+0000", 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041869811 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571063+0000", 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6340999999999994e-05 2024-09-16T11:45:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.571108+0000", 2024-09-16T11:45:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5336000000000001e-05 2024-09-16T11:45:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19580 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 8036352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:45:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.495316922000001, 2024-09-16T11:45:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043451596000000002, 2024-09-16T11:45:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19580 2024-09-16T11:45:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:45:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.529297+0000", 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531106+0000", 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001808179 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531110+0000", 2024-09-16T11:45:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5990000000000004e-06 2024-09-16T11:45:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531123+0000", 2024-09-16T11:45:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2687000000000001e-05 2024-09-16T11:45:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531129+0000", 2024-09-16T11:45:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:45:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572642+0000", 2024-09-16T11:45:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041513474000000002 2024-09-16T11:45:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572691+0000", 2024-09-16T11:45:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8989000000000001e-05 2024-09-16T11:45:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.572749+0000", 2024-09-16T11:45:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7806999999999997e-05 2024-09-16T11:45:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19709 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4849664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:45:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.493256880999994, 2024-09-16T11:45:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043537291999999998, 2024-09-16T11:45:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19709 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531357+0000", 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531358+0000", 2024-09-16T11:45:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:45:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531360+0000", 2024-09-16T11:45:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:45:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531365+0000", 2024-09-16T11:45:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.716e-06 2024-09-16T11:45:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531369+0000", 2024-09-16T11:45:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5410000000000001e-06 2024-09-16T11:45:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574763+0000", 2024-09-16T11:45:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043394359 2024-09-16T11:45:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574831+0000", 2024-09-16T11:45:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7997999999999995e-05 2024-09-16T11:45:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574895+0000", 2024-09-16T11:45:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3427000000000001e-05 2024-09-16T11:45:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19710 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4956160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:45:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.493241601999998, 2024-09-16T11:45:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043674198999999997, 2024-09-16T11:45:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19710 2024-09-16T11:45:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.531373+0000", 2024-09-16T11:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533105+0000", 2024-09-16T11:45:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001731961 2024-09-16T11:45:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533109+0000", 2024-09-16T11:45:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3080000000000001e-06 2024-09-16T11:45:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533120+0000", 2024-09-16T11:45:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0625000000000001e-05 2024-09-16T11:45:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533125+0000", 2024-09-16T11:45:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:45:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574923+0000", 2024-09-16T11:45:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041797187 2024-09-16T11:45:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.574991+0000", 2024-09-16T11:45:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8752999999999997e-05 2024-09-16T11:45:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.575047+0000", 2024-09-16T11:45:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5596000000000001e-05 2024-09-16T11:45:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19719 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5488640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:45:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.491269853000006, 2024-09-16T11:45:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043412013999999999, 2024-09-16T11:45:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19719 2024-09-16T11:45:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533344+0000", 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533345+0000", 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533347+0000", 2024-09-16T11:45:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8139999999999999e-06 2024-09-16T11:45:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533352+0000", 2024-09-16T11:45:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4970000000000001e-06 2024-09-16T11:45:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533355+0000", 2024-09-16T11:45:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.711e-06 2024-09-16T11:45:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576629+0000", 2024-09-16T11:45:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043274295999999997 2024-09-16T11:45:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576706+0000", 2024-09-16T11:45:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6711000000000005e-05 2024-09-16T11:45:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576757+0000", 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0509e-05 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19720 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 5808128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.491257203999993, 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043577006000000001, 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19720 2024-09-16T11:45:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:45:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:45:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533357+0000", 2024-09-16T11:45:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.35e-07 2024-09-16T11:45:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533359+0000", 2024-09-16T11:45:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:45:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533364+0000", 2024-09-16T11:45:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4349999999999996e-06 2024-09-16T11:45:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533367+0000", 2024-09-16T11:45:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2349999999999999e-06 2024-09-16T11:45:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576786+0000", 2024-09-16T11:45:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043418885999999997 2024-09-16T11:45:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576862+0000", 2024-09-16T11:45:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6172000000000002e-05 2024-09-16T11:45:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576934+0000", 2024-09-16T11:45:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1891999999999995e-05 2024-09-16T11:45:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19721 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6234112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:45:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.491243683999997, 2024-09-16T11:45:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043681656999999999, 2024-09-16T11:45:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19721 2024-09-16T11:45:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:45:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:45:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533371+0000", 2024-09-16T11:45:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1e-07 2024-09-16T11:45:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533373+0000", 2024-09-16T11:45:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.973e-06 2024-09-16T11:45:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533377+0000", 2024-09-16T11:45:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.403e-06 2024-09-16T11:45:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533379+0000", 2024-09-16T11:45:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.058e-06 2024-09-16T11:45:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.576959+0000", 2024-09-16T11:45:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043579433000000001 2024-09-16T11:45:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577015+0000", 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5859000000000002e-05 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577052+0000", 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7620999999999998e-05 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.093 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19722 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6340608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.093 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:45:00.093 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.491232956999994, 2024-09-16T11:45:00.093 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043854922999999997, 2024-09-16T11:45:00.093 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.093 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.093 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.093 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.093 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.093 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19722 2024-09-16T11:45:00.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.093 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.094 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:45:00.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.094 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.533381+0000", 2024-09-16T11:45:00.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.094 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535103+0000", 2024-09-16T11:45:00.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017217020000000001 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535107+0000", 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2819999999999998e-06 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535118+0000", 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0148e-05 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535123+0000", 2024-09-16T11:45:00.096 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:45:00.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.096 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.096 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577091+0000", 2024-09-16T11:45:00.096 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041967930000000001 2024-09-16T11:45:00.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.096 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.096 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577163+0000", 2024-09-16T11:45:00.096 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1853999999999993e-05 2024-09-16T11:45:00.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.097 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.097 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.577236+0000", 2024-09-16T11:45:00.097 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3699e-05 2024-09-16T11:45:00.097 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.097 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.097 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.097 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19732 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6447104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.097 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:45:00.097 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.489373100999998, 2024-09-16T11:45:00.097 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043399048000000003, 2024-09-16T11:45:00.097 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19732 2024-09-16T11:45:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:45:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.099 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.099 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535241+0000", 2024-09-16T11:45:00.099 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.099 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.099 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535242+0000", 2024-09-16T11:45:00.099 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:45:00.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.099 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.099 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535243+0000", 2024-09-16T11:45:00.099 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.793e-06 2024-09-16T11:45:00.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.100 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.100 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535249+0000", 2024-09-16T11:45:00.100 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.643e-06 2024-09-16T11:45:00.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.100 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.100 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535252+0000", 2024-09-16T11:45:00.100 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:45:00.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.100 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.101 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578501+0000", 2024-09-16T11:45:00.101 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043248343000000002 2024-09-16T11:45:00.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.101 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.101 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578576+0000", 2024-09-16T11:45:00.101 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5544999999999996e-05 2024-09-16T11:45:00.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.101 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.101 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578640+0000", 2024-09-16T11:45:00.101 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4018999999999998e-05 2024-09-16T11:45:00.101 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19733 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 6873088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.489352260000004, 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043549681, 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.102 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19733 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535262+0000", 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.104 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535263+0000", 2024-09-16T11:45:00.104 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:45:00.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.104 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.104 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535265+0000", 2024-09-16T11:45:00.104 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9369999999999998e-06 2024-09-16T11:45:00.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.104 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.104 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535271+0000", 2024-09-16T11:45:00.104 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9719999999999997e-06 2024-09-16T11:45:00.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535274+0000", 2024-09-16T11:45:00.105 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:45:00.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578668+0000", 2024-09-16T11:45:00.105 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043393355000000002 2024-09-16T11:45:00.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578744+0000", 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6853000000000004e-05 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578812+0000", 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.737e-05 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19734 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 7299072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:45:00.106 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.489336512999998, 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043699189999999999, 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19734 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.535278+0000", 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537101+0000", 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001823098 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537105+0000", 2024-09-16T11:45:00.108 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:45:00.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.109 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.109 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537119+0000", 2024-09-16T11:45:00.109 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614999999999999e-05 2024-09-16T11:45:00.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.109 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.109 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537125+0000", 2024-09-16T11:45:00.109 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:45:00.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.109 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578838+0000", 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041712605999999999 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578912+0000", 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4396999999999999e-05 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.578977+0000", 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4858999999999997e-05 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.110 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.111 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.111 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19851 4.0 4:070f16ce:::rbd_data.11c1a7dbfac1.0000000000000002:f [read 4431872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.111 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:45:00.111 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.487255769000001, 2024-09-16T11:45:00.111 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043564540999999998, 2024-09-16T11:45:00.111 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.111 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:00.111 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.111 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.111 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.111 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19851 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.537359+0000", 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.112 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539100+0000", 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017412079999999999 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539105+0000", 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539116+0000", 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1824999999999999e-05 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.539122+0000", 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9859999999999999e-06 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580764+0000", 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.041641299 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580839+0000", 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5167999999999996e-05 2024-09-16T11:45:00.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.115 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.115 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.580923+0000", 2024-09-16T11:45:00.115 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4314999999999995e-05 2024-09-16T11:45:00.115 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.115 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.115 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.115 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:55904 4.14 4:2d2dfe38:::rbd_data.11c1b01b799a.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2068993~28159] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T11:45:00.115 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:45:00.115 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 31.932878869, 2024-09-16T11:45:00.115 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.90010917700000004, 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 55904 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091735+0000", 2024-09-16T11:45:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091737+0000", 2024-09-16T11:45:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.254e-06 2024-09-16T11:45:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091759+0000", 2024-09-16T11:45:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2487999999999999e-05 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091769+0000", 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0199e-05 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091774+0000", 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.118 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091830+0000", 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6700999999999998e-05 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.091840+0000", 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0128e-05 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990355+0000", 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.89851413099999999 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.119 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:00.120 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990509+0000", 2024-09-16T11:45:00.120 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000154442 2024-09-16T11:45:00.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.120 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:45:00.120 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.990626+0000", 2024-09-16T11:45:00.120 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000116928 2024-09-16T11:45:00.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.120 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:45:00.120 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991526+0000", 2024-09-16T11:45:00.120 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00089961800000000003 2024-09-16T11:45:00.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991793+0000", 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026779000000000001 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991824+0000", 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0380999999999999e-05 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:28.991845+0000", 2024-09-16T11:45:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0919000000000001e-05 2024-09-16T11:45:00.122 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.122 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.122 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.122 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:00.122 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:00.122 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:00.123 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:45:00.124 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:00.121+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:00.224 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:00.223+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:00.234 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:00.234 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:00.234 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:00.234 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:00.243 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:45:00.325 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:45:00.323+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:45:00.342 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:00.342 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:00.342 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:00.342 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:00.342 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:00.350 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:45:00.425 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:00.424+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:00.450 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:00.450 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:00.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:00.450 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:00.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.450 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.450 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:00.450 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.086232379999998, 2024-09-16T11:45:00.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:45:00.451 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.451 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.451 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:45:00.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.451 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:00.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:00.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:00.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:45:00.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:45:00.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:45:00.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:45:00.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:45:00.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:45:00.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:45:00.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:45:00.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.455 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.455 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.455 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:00.455 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.086210351000005, 2024-09-16T11:45:00.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:45:00.455 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.455 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.456 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:45:00.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.456 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:00.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:00.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:00.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:45:00.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:45:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:45:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:45:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:45:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:45:00.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:45:00.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:45:00.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:45:00.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:45:00.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:45:00.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:45:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:45:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.086195586000002, 2024-09-16T11:45:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:45:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:45:00.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:45:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:45:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:45:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:45:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:45:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:45:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:45:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:45:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:45:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:45:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:45:00.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.086175518000005, 2024-09-16T11:45:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:45:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:45:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:45:00.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:45:00.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:45:00.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:45:00.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:45:00.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:45:00.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.469 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.086144681999997, 2024-09-16T11:45:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:45:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:45:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:45:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:45:00.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:45:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:45:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:45:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:45:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:45:00.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:45:00.474 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.474 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.474 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.086125162000002, 2024-09-16T11:45:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:45:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:45:00.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:45:00.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.086097582999997, 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:45:00.479 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:45:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:45:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:45:00.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:45:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:45:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:45:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:45:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:45:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.086078888000003, 2024-09-16T11:45:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:45:00.484 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:45:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:45:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:45:00.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:45:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:45:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:45:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:45:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:45:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:45:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.488 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.086063878999994, 2024-09-16T11:45:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:45:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.489 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:45:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:45:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:45:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:45:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:45:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:45:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:45:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:45:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:45:00.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:45:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:45:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:45:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:45:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.086031781000003, 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:45:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:45:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:45:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:45:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:45:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:45:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:45:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:45:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:45:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:45:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:45:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:45:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:45:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:45:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.502 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.086008161999999, 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.503 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:45:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:45:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:45:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:45:00.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:45:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:45:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:45:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:45:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:45:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.507 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.085993432999999, 2024-09-16T11:45:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:45:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:45:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:45:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:45:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:45:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:45:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:45:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:45:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:45:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:45:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:45:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:45:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:45:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:45:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.513 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.085971608999998, 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:45:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:45:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:45:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:45:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:45:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:45:00.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:45:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:45:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:45:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:45:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.085956959000001, 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:45:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:00.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:45:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:45:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:45:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:45:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:45:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:45:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:45:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:45:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:45:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:45:00.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:45:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:45:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:45:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:45:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.085935386000003, 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:00.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:45:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:45:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:45:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:45:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:45:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.085915940000007, 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.527 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:45:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:45:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:45:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:45:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:45:00.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:45:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:45:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:45:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:45:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.085879242999994, 2024-09-16T11:45:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:45:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:45:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.532 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:45:00.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:45:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:45:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:45:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:45:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:45:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:45:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.085857297000004, 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.536 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:45:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:45:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:45:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:45:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:45:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:45:00.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.085835411000005, 2024-09-16T11:45:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:45:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:45:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.541 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:00.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:45:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:45:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.085820691999999, 2024-09-16T11:45:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:45:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.545 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:45:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:45:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:45:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:45:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:45:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:45:00.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.549 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.550 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:00.551 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T11:45:00.552 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:45:00.552 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:00.548+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:00.652 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:00.650+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:00.691 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:00.691 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:00.691 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:00.692 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:00.700 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:45:00.753 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:45:00.751+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:45:00.797 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:00.798 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:00.798 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:00.798 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:00.798 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:00.806 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:45:00.853 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:00.851+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:00.905 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19943 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2416640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:45:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376761505000005, 2024-09-16T11:45:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027947257999999999, 2024-09-16T11:45:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19943 2024-09-16T11:45:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:45:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:45:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:45:00.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5999999999999999e-07 2024-09-16T11:45:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526297+0000", 2024-09-16T11:45:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8059999999999999e-06 2024-09-16T11:45:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526303+0000", 2024-09-16T11:45:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6579999999999999e-06 2024-09-16T11:45:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526306+0000", 2024-09-16T11:45:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3969999999999998e-06 2024-09-16T11:45:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554165+0000", 2024-09-16T11:45:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027858519000000002 2024-09-16T11:45:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554201+0000", 2024-09-16T11:45:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5908000000000002e-05 2024-09-16T11:45:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554242+0000", 2024-09-16T11:45:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1510000000000001e-05 2024-09-16T11:45:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19944 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2842624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:45:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376746609999998, 2024-09-16T11:45:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028052639000000001, 2024-09-16T11:45:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19944 2024-09-16T11:45:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:45:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:45:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:45:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:45:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526312+0000", 2024-09-16T11:45:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:45:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526317+0000", 2024-09-16T11:45:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5910000000000003e-06 2024-09-16T11:45:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526321+0000", 2024-09-16T11:45:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2339999999999999e-06 2024-09-16T11:45:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554263+0000", 2024-09-16T11:45:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027942181 2024-09-16T11:45:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554307+0000", 2024-09-16T11:45:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4499999999999997e-05 2024-09-16T11:45:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554362+0000", 2024-09-16T11:45:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5078999999999999e-05 2024-09-16T11:45:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19945 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3162112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:45:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376732055000005, 2024-09-16T11:45:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028145673999999999, 2024-09-16T11:45:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19945 2024-09-16T11:45:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:45:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:45:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526325+0000", 2024-09-16T11:45:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:45:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526326+0000", 2024-09-16T11:45:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.733e-06 2024-09-16T11:45:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526332+0000", 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526336+0000", 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8410000000000002e-06 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554389+0000", 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028053576 2024-09-16T11:45:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554427+0000", 2024-09-16T11:45:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7299999999999999e-05 2024-09-16T11:45:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554470+0000", 2024-09-16T11:45:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3195000000000002e-05 2024-09-16T11:45:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19946 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3268608~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:45:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376716475999999, 2024-09-16T11:45:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028232587, 2024-09-16T11:45:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19946 2024-09-16T11:45:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:45:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:45:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526346+0000", 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6459999999999998e-06 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526352+0000", 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1070000000000001e-06 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526355+0000", 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554493+0000", 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.02813746 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554530+0000", 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6881999999999997e-05 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554572+0000", 2024-09-16T11:45:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2688999999999997e-05 2024-09-16T11:45:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19947 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4014080~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:45:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376697121999996, 2024-09-16T11:45:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028315646999999999, 2024-09-16T11:45:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19947 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:45:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526360+0000", 2024-09-16T11:45:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.01e-07 2024-09-16T11:45:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526361+0000", 2024-09-16T11:45:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8300000000000001e-06 2024-09-16T11:45:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526369+0000", 2024-09-16T11:45:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:45:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526377+0000", 2024-09-16T11:45:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0909999999999993e-06 2024-09-16T11:45:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554593+0000", 2024-09-16T11:45:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028216657999999999 2024-09-16T11:45:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554632+0000", 2024-09-16T11:45:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8828000000000002e-05 2024-09-16T11:45:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554675+0000", 2024-09-16T11:45:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2710000000000003e-05 2024-09-16T11:45:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19948 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3588096~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:45:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376674722999994, 2024-09-16T11:45:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028394545, 2024-09-16T11:45:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19948 2024-09-16T11:45:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:45:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:45:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:45:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9200000000000001e-07 2024-09-16T11:45:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526384+0000", 2024-09-16T11:45:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:45:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526391+0000", 2024-09-16T11:45:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5150000000000003e-06 2024-09-16T11:45:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526394+0000", 2024-09-16T11:45:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4240000000000002e-06 2024-09-16T11:45:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554695+0000", 2024-09-16T11:45:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028301218999999999 2024-09-16T11:45:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554731+0000", 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6298999999999999e-05 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554776+0000", 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4675999999999997e-05 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19949 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1351680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:45:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376658450999997, 2024-09-16T11:45:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028490081, 2024-09-16T11:45:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19949 2024-09-16T11:45:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526400+0000", 2024-09-16T11:45:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7099999999999999e-06 2024-09-16T11:45:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526406+0000", 2024-09-16T11:45:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5140000000000001e-06 2024-09-16T11:45:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526409+0000", 2024-09-16T11:45:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5360000000000001e-06 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554801+0000", 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028392378999999999 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554843+0000", 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1056000000000001e-05 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554888+0000", 2024-09-16T11:45:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466999999999997e-05 2024-09-16T11:45:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19950 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1777664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:45:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376643455000007, 2024-09-16T11:45:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028661844999999998, 2024-09-16T11:45:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19950 2024-09-16T11:45:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:45:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:45:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:45:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:45:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526415+0000", 2024-09-16T11:45:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.776e-06 2024-09-16T11:45:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526422+0000", 2024-09-16T11:45:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6930000000000001e-06 2024-09-16T11:45:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526425+0000", 2024-09-16T11:45:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:45:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554923+0000", 2024-09-16T11:45:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028498102000000001 2024-09-16T11:45:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555000+0000", 2024-09-16T11:45:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6619999999999995e-05 2024-09-16T11:45:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555075+0000", 2024-09-16T11:45:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4931999999999993e-05 2024-09-16T11:45:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19951 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2097152~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:45:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376627643999996, 2024-09-16T11:45:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028755718, 2024-09-16T11:45:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19951 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:45:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-16T11:45:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526431+0000", 2024-09-16T11:45:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:45:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526441+0000", 2024-09-16T11:45:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7790000000000007e-06 2024-09-16T11:45:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526445+0000", 2024-09-16T11:45:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.634e-06 2024-09-16T11:45:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555100+0000", 2024-09-16T11:45:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654722000000001 2024-09-16T11:45:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555138+0000", 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7996e-05 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555184+0000", 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6427999999999997e-05 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19952 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2203648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376607084, 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028901157, 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19952 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:45:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:45:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526450+0000", 2024-09-16T11:45:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:45:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526452+0000", 2024-09-16T11:45:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9039999999999999e-06 2024-09-16T11:45:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526465+0000", 2024-09-16T11:45:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3559e-05 2024-09-16T11:45:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526470+0000", 2024-09-16T11:45:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555217+0000", 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028747287999999999 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555272+0000", 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4885000000000001e-05 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555350+0000", 2024-09-16T11:45:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8428000000000002e-05 2024-09-16T11:45:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19953 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2523136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:45:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376582491999997, 2024-09-16T11:45:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029004309999999998, 2024-09-16T11:45:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19953 2024-09-16T11:45:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:45:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:45:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:45:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-16T11:45:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526476+0000", 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9810000000000002e-06 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526482+0000", 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526485+0000", 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2760000000000001e-06 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555383+0000", 2024-09-16T11:45:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028898113 2024-09-16T11:45:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555433+0000", 2024-09-16T11:45:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9753999999999998e-05 2024-09-16T11:45:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555478+0000", 2024-09-16T11:45:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5006999999999998e-05 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19954 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2949120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376567495000003, 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029165054999999999, 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19954 2024-09-16T11:45:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:45:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:45:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:45:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1799999999999995e-07 2024-09-16T11:45:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526491+0000", 2024-09-16T11:45:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7320000000000001e-06 2024-09-16T11:45:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526497+0000", 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6869999999999999e-06 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526500+0000", 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555513+0000", 2024-09-16T11:45:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029012626 2024-09-16T11:45:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555576+0000", 2024-09-16T11:45:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3288999999999997e-05 2024-09-16T11:45:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555654+0000", 2024-09-16T11:45:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7861999999999994e-05 2024-09-16T11:45:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19955 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3375104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376548193000005, 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029241979000000001, 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19955 2024-09-16T11:45:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526509+0000", 2024-09-16T11:45:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-09-16T11:45:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526510+0000", 2024-09-16T11:45:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.844e-06 2024-09-16T11:45:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526516+0000", 2024-09-16T11:45:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9039999999999997e-06 2024-09-16T11:45:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526520+0000", 2024-09-16T11:45:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3639999999999999e-06 2024-09-16T11:45:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555669+0000", 2024-09-16T11:45:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029149628 2024-09-16T11:45:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555706+0000", 2024-09-16T11:45:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7017000000000002e-05 2024-09-16T11:45:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555750+0000", 2024-09-16T11:45:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3856999999999997e-05 2024-09-16T11:45:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19956 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3694592~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:45:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376533237999993, 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029398799999999999, 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19956 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:45:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526524+0000", 2024-09-16T11:45:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:45:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526525+0000", 2024-09-16T11:45:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-16T11:45:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526531+0000", 2024-09-16T11:45:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6010000000000002e-06 2024-09-16T11:45:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526535+0000", 2024-09-16T11:45:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.597e-06 2024-09-16T11:45:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555785+0000", 2024-09-16T11:45:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029250235999999999 2024-09-16T11:45:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555845+0000", 2024-09-16T11:45:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0393000000000003e-05 2024-09-16T11:45:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555922+0000", 2024-09-16T11:45:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6710000000000002e-05 2024-09-16T11:45:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19957 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4120576~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:45:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.376518085000001, 2024-09-16T11:45:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029481157000000001, 2024-09-16T11:45:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19957 2024-09-16T11:45:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:45:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:45:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526539+0000", 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526540+0000", 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.759e-06 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526546+0000", 2024-09-16T11:45:00.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1079999999999998e-06 2024-09-16T11:45:00.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526550+0000", 2024-09-16T11:45:00.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.269e-06 2024-09-16T11:45:00.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555939+0000", 2024-09-16T11:45:00.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029389444000000001 2024-09-16T11:45:00.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555977+0000", 2024-09-16T11:45:00.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7487999999999998e-05 2024-09-16T11:45:00.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.556019+0000", 2024-09-16T11:45:00.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2756999999999997e-05 2024-09-16T11:45:00.978 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.978 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.978 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.979 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:28959 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:45:00.979 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:45:00.979 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.893661672999997, 2024-09-16T11:45:00.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031145285000000002, 2024-09-16T11:45:00.979 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.979 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:00.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.979 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 28959 2024-09-16T11:45:00.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.979 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009396+0000", 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.296e-06 2024-09-16T11:45:00.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030864+0000", 2024-09-16T11:45:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.021468451 2024-09-16T11:45:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030892+0000", 2024-09-16T11:45:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7555999999999998e-05 2024-09-16T11:45:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030902+0000", 2024-09-16T11:45:00.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9550000000000004e-06 2024-09-16T11:45:00.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030979+0000", 2024-09-16T11:45:00.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7137000000000004e-05 2024-09-16T11:45:00.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031051+0000", 2024-09-16T11:45:00.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1475000000000003e-05 2024-09-16T11:45:00.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:45:00.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031389+0000", 2024-09-16T11:45:00.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033827100000000001 2024-09-16T11:45:00.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:00.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.039264+0000", 2024-09-16T11:45:00.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078751700000000008 2024-09-16T11:45:00.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:00.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040484+0000", 2024-09-16T11:45:00.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012199579999999999 2024-09-16T11:45:00.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:00.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040517+0000", 2024-09-16T11:45:00.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2960999999999998e-05 2024-09-16T11:45:00.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040540+0000", 2024-09-16T11:45:00.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3054999999999999e-05 2024-09-16T11:45:00.984 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.984 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.984 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30605 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 3915776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.659355413999997, 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938655999999999, 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30605 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.985 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:45:00.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:45:00.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:45:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:45:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243704+0000", 2024-09-16T11:45:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2199999999999999e-06 2024-09-16T11:45:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243710+0000", 2024-09-16T11:45:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6340000000000003e-06 2024-09-16T11:45:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243714+0000", 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5030000000000002e-06 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271450+0000", 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027736035999999999 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271536+0000", 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6521999999999996e-05 2024-09-16T11:45:00.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271640+0000", 2024-09-16T11:45:00.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103341 2024-09-16T11:45:00.989 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.989 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.989 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.989 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30607 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4046848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:45:00.989 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:45:00.989 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.659332259000003, 2024-09-16T11:45:00.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028130446, 2024-09-16T11:45:00.990 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.990 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.990 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.990 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30607 2024-09-16T11:45:00.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.990 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:45:00.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:45:00.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243725+0000", 2024-09-16T11:45:00.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2399999999999998e-07 2024-09-16T11:45:00.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243727+0000", 2024-09-16T11:45:00.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3379999999999999e-06 2024-09-16T11:45:00.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243733+0000", 2024-09-16T11:45:00.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4280000000000001e-06 2024-09-16T11:45:00.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243737+0000", 2024-09-16T11:45:00.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8990000000000003e-06 2024-09-16T11:45:00.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271676+0000", 2024-09-16T11:45:00.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938481000000001 2024-09-16T11:45:00.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271757+0000", 2024-09-16T11:45:00.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1396999999999993e-05 2024-09-16T11:45:00.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271855+0000", 2024-09-16T11:45:00.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7379000000000004e-05 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30608 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.659315407999998, 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028323879, 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.995 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30608 2024-09-16T11:45:00.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.995 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:45:00.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:00.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:45:00.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:00.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:00.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:45:00.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:45:00.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:00.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243743+0000", 2024-09-16T11:45:00.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.762e-06 2024-09-16T11:45:00.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:00.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243749+0000", 2024-09-16T11:45:00.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5670000000000002e-06 2024-09-16T11:45:00.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:00.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243758+0000", 2024-09-16T11:45:00.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8309999999999996e-06 2024-09-16T11:45:00.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:00.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271879+0000", 2024-09-16T11:45:00.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028120501999999999 2024-09-16T11:45:00.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271965+0000", 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5755999999999997e-05 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.272065+0000", 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000100101 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.998 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:56045 4.15 4:ab2ce1b7:::rbd_data.11c1b01b799a.0000000000000020:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.724032157, 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032248165000000002, 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 56045 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:00.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:45:01.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:45:01.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179026+0000", 2024-09-16T11:45:01.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.556e-06 2024-09-16T11:45:01.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179032+0000", 2024-09-16T11:45:01.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5379999999999999e-06 2024-09-16T11:45:01.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179045+0000", 2024-09-16T11:45:01.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.277e-05 2024-09-16T11:45:01.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179067+0000", 2024-09-16T11:45:01.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.243e-05 2024-09-16T11:45:01.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179120+0000", 2024-09-16T11:45:01.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2726000000000003e-05 2024-09-16T11:45:01.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179279+0000", 2024-09-16T11:45:01.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015865899999999999 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.185163+0000", 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0058837639999999997 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211197+0000", 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026034130999999999 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:01.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211206+0000", 2024-09-16T11:45:01.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2539999999999998e-06 2024-09-16T11:45:01.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211245+0000", 2024-09-16T11:45:01.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8924999999999997e-05 2024-09-16T11:45:01.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211272+0000", 2024-09-16T11:45:01.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7412000000000001e-05 2024-09-16T11:45:01.004 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.005 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.005 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.005 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.005 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.005 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:01.006 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:45:01.007 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:01.004+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:01.106 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:01.106 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:01.106 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:01.106 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:01.107 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:01.105+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:01.114 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:45:01.208 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:01.206+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:01.211 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:01.211 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:01.211 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:01.212 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:01.212 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:01.220 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:45:01.308 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:01.307+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:01.319 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19282 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:45:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.843683718999998, 2024-09-16T11:45:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521151000000002, 2024-09-16T11:45:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19282 2024-09-16T11:45:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:45:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:45:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472838+0000", 2024-09-16T11:45:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:45:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472848+0000", 2024-09-16T11:45:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9799999999999993e-06 2024-09-16T11:45:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472858+0000", 2024-09-16T11:45:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9429999999999995e-06 2024-09-16T11:45:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472862+0000", 2024-09-16T11:45:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4460000000000003e-06 2024-09-16T11:45:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501277+0000", 2024-09-16T11:45:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028414900999999999 2024-09-16T11:45:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501322+0000", 2024-09-16T11:45:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4880999999999999e-05 2024-09-16T11:45:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501358+0000", 2024-09-16T11:45:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6245000000000003e-05 2024-09-16T11:45:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19286 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7979008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:45:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.843310786999993, 2024-09-16T11:45:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521860999999999, 2024-09-16T11:45:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19286 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:45:01.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473211+0000", 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473214+0000", 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1080000000000001e-06 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473226+0000", 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2441e-05 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473234+0000", 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729999999999998e-06 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501592+0000", 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028358491999999999 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501671+0000", 2024-09-16T11:45:01.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8846999999999999e-05 2024-09-16T11:45:01.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501732+0000", 2024-09-16T11:45:01.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1118000000000007e-05 2024-09-16T11:45:01.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.328 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.329 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20619 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7610368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:01.329 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:45:01.329 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.755961619000004, 2024-09-16T11:45:01.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028500152000000001, 2024-09-16T11:45:01.329 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.329 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.329 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20619 2024-09-16T11:45:01.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.329 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:45:01.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:45:01.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560560+0000", 2024-09-16T11:45:01.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9499999999999998e-07 2024-09-16T11:45:01.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560561+0000", 2024-09-16T11:45:01.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.071e-06 2024-09-16T11:45:01.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560565+0000", 2024-09-16T11:45:01.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:45:01.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560567+0000", 2024-09-16T11:45:01.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:45:01.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588949+0000", 2024-09-16T11:45:01.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381878999999999 2024-09-16T11:45:01.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588989+0000", 2024-09-16T11:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0649000000000003e-05 2024-09-16T11:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589059+0000", 2024-09-16T11:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0239000000000003e-05 2024-09-16T11:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.333 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20620 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7716864~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:01.334 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:45:01.334 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.755952218999994, 2024-09-16T11:45:01.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028627621999999998, 2024-09-16T11:45:01.334 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.334 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.334 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.334 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.334 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.334 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20620 2024-09-16T11:45:01.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.334 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:45:01.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:45:01.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562377+0000", 2024-09-16T11:45:01.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018081729999999999 2024-09-16T11:45:01.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562380+0000", 2024-09-16T11:45:01.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0469999999999998e-06 2024-09-16T11:45:01.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562386+0000", 2024-09-16T11:45:01.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2040000000000002e-06 2024-09-16T11:45:01.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562390+0000", 2024-09-16T11:45:01.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:45:01.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589092+0000", 2024-09-16T11:45:01.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026701798999999998 2024-09-16T11:45:01.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589139+0000", 2024-09-16T11:45:01.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7320000000000001e-05 2024-09-16T11:45:01.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589196+0000", 2024-09-16T11:45:01.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7201999999999999e-05 2024-09-16T11:45:01.338 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.338 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.338 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41954 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.902246429000002, 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028594441000000002, 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41954 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:45:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:45:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414275+0000", 2024-09-16T11:45:01.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:45:01.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414277+0000", 2024-09-16T11:45:01.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:45:01.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414282+0000", 2024-09-16T11:45:01.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7559999999999996e-06 2024-09-16T11:45:01.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414286+0000", 2024-09-16T11:45:01.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:45:01.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442778+0000", 2024-09-16T11:45:01.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028492237 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442816+0000", 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8198000000000003e-05 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442869+0000", 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2589000000000001e-05 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.344 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41955 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.344 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:01.344 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.902231370999999, 2024-09-16T11:45:01.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028687865, 2024-09-16T11:45:01.344 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.344 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.344 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41955 2024-09-16T11:45:01.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.345 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:01.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:01.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414292+0000", 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414299+0000", 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4279999999999999e-06 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414303+0000", 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442892+0000", 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028589564000000001 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442930+0000", 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330999999999999e-05 2024-09-16T11:45:01.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442977+0000", 2024-09-16T11:45:01.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7874000000000003e-05 2024-09-16T11:45:01.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.348 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41956 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:45:01.348 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.902196054000001, 2024-09-16T11:45:01.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028766896, 2024-09-16T11:45:01.349 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.349 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.349 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41956 2024-09-16T11:45:01.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.349 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:45:01.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414326+0000", 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9900000000000009e-07 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414330+0000", 2024-09-16T11:45:01.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2339999999999996e-06 2024-09-16T11:45:01.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414341+0000", 2024-09-16T11:45:01.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1358999999999999e-05 2024-09-16T11:45:01.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414347+0000", 2024-09-16T11:45:01.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8100000000000003e-06 2024-09-16T11:45:01.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443002+0000", 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654331000000002 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443040+0000", 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8262000000000001e-05 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443092+0000", 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1901000000000003e-05 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.352 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.353 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41957 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.353 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:45:01.353 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.902168635000002, 2024-09-16T11:45:01.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028848318000000001, 2024-09-16T11:45:01.353 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.353 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.353 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41957 2024-09-16T11:45:01.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.354 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:45:01.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:45:01.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414353+0000", 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414355+0000", 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3489999999999999e-06 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414364+0000", 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1449999999999997e-06 2024-09-16T11:45:01.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414368+0000", 2024-09-16T11:45:01.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8940000000000003e-06 2024-09-16T11:45:01.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443115+0000", 2024-09-16T11:45:01.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028746532000000002 2024-09-16T11:45:01.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443152+0000", 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6814999999999999e-05 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443201+0000", 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9069000000000001e-05 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41958 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:45:01.357 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.902144853000003, 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028939856, 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41958 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414377+0000", 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4099999999999999e-07 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414379+0000", 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.418e-06 2024-09-16T11:45:01.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414385+0000", 2024-09-16T11:45:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999999e-06 2024-09-16T11:45:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414389+0000", 2024-09-16T11:45:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7740000000000002e-06 2024-09-16T11:45:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443223+0000", 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028833919 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443263+0000", 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0148999999999998e-05 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443316+0000", 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2636000000000002e-05 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41959 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.902127927999999, 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029040309, 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41959 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.362 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:45:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414395+0000", 2024-09-16T11:45:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7579999999999999e-06 2024-09-16T11:45:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414401+0000", 2024-09-16T11:45:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5330000000000002e-06 2024-09-16T11:45:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414404+0000", 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.529e-06 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443339+0000", 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028934933999999999 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443383+0000", 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3832000000000003e-05 2024-09-16T11:45:01.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443433+0000", 2024-09-16T11:45:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0309000000000002e-05 2024-09-16T11:45:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41960 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.902113094000001, 2024-09-16T11:45:01.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029138134, 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41960 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-07 2024-09-16T11:45:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414410+0000", 2024-09-16T11:45:01.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:45:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414417+0000", 2024-09-16T11:45:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:45:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414420+0000", 2024-09-16T11:45:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:45:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443457+0000", 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029037065000000001 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443495+0000", 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7555000000000002e-05 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443546+0000", 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1422000000000003e-05 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.370 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41961 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.902097112999996, 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029230973, 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41961 2024-09-16T11:45:01.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4499999999999998e-07 2024-09-16T11:45:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414432+0000", 2024-09-16T11:45:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.413e-06 2024-09-16T11:45:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414439+0000", 2024-09-16T11:45:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4769999999999997e-06 2024-09-16T11:45:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414443+0000", 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8859999999999997e-06 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443569+0000", 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029126124 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443606+0000", 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6588999999999998e-05 2024-09-16T11:45:01.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443655+0000", 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9138999999999998e-05 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41962 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.902074145999997, 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029324505000000001, 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.375 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41962 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:45:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414449+0000", 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0279999999999999e-06 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414456+0000", 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3339999999999998e-06 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414459+0000", 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.382e-06 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443678+0000", 2024-09-16T11:45:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029218958999999999 2024-09-16T11:45:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443719+0000", 2024-09-16T11:45:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0722000000000001e-05 2024-09-16T11:45:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443771+0000", 2024-09-16T11:45:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2661000000000003e-05 2024-09-16T11:45:01.379 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41963 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.902058242999999, 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029416557999999999, 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.380 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41963 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414465+0000", 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414470+0000", 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4750000000000001e-06 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414474+0000", 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3720000000000001e-06 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443795+0000", 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029320821 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443830+0000", 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5463000000000002e-05 2024-09-16T11:45:01.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443879+0000", 2024-09-16T11:45:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9147000000000002e-05 2024-09-16T11:45:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41964 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.902043355000004, 2024-09-16T11:45:01.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029521202999999999, 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41964 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414480+0000", 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:45:01.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414486+0000", 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414489+0000", 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2770000000000001e-06 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443903+0000", 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029414109000000001 2024-09-16T11:45:01.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443944+0000", 2024-09-16T11:45:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0992999999999999e-05 2024-09-16T11:45:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443999+0000", 2024-09-16T11:45:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4632000000000001e-05 2024-09-16T11:45:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41965 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.902028393000002, 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029611293, 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41965 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:45:01.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414495+0000", 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.88e-06 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414501+0000", 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2489999999999998e-06 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414504+0000", 2024-09-16T11:45:01.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3139999999999999e-06 2024-09-16T11:45:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444023+0000", 2024-09-16T11:45:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029518564000000001 2024-09-16T11:45:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444058+0000", 2024-09-16T11:45:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5196e-05 2024-09-16T11:45:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444104+0000", 2024-09-16T11:45:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5732999999999997e-05 2024-09-16T11:45:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41966 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.902012186999997, 2024-09-16T11:45:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029707678000000001, 2024-09-16T11:45:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41966 2024-09-16T11:45:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:45:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414514+0000", 2024-09-16T11:45:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-16T11:45:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414518+0000", 2024-09-16T11:45:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3839999999999999e-06 2024-09-16T11:45:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414521+0000", 2024-09-16T11:45:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0400000000000001e-06 2024-09-16T11:45:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444128+0000", 2024-09-16T11:45:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029606612000000001 2024-09-16T11:45:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444167+0000", 2024-09-16T11:45:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9418000000000001e-05 2024-09-16T11:45:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444216+0000", 2024-09-16T11:45:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9354999999999998e-05 2024-09-16T11:45:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41967 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.901996378, 2024-09-16T11:45:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029796916, 2024-09-16T11:45:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.399 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41967 2024-09-16T11:45:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6699999999999999e-07 2024-09-16T11:45:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414527+0000", 2024-09-16T11:45:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7060000000000001e-06 2024-09-16T11:45:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414533+0000", 2024-09-16T11:45:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2140000000000001e-06 2024-09-16T11:45:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414536+0000", 2024-09-16T11:45:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:45:01.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444239+0000", 2024-09-16T11:45:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029702698999999999 2024-09-16T11:45:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444273+0000", 2024-09-16T11:45:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4229e-05 2024-09-16T11:45:01.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444321+0000", 2024-09-16T11:45:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8477999999999999e-05 2024-09-16T11:45:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41968 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:45:01.404 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.901981481, 2024-09-16T11:45:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029902765000000001, 2024-09-16T11:45:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41968 2024-09-16T11:45:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:45:01.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414540+0000", 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414542+0000", 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414547+0000", 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4639999999999997e-06 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414551+0000", 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6279999999999998e-06 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444346+0000", 2024-09-16T11:45:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029794885 2024-09-16T11:45:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444389+0000", 2024-09-16T11:45:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3442000000000002e-05 2024-09-16T11:45:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444442+0000", 2024-09-16T11:45:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3173000000000001e-05 2024-09-16T11:45:01.408 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41969 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.901966569000002, 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029997612999999999, 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:01.409 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41969 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414555+0000", 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414556+0000", 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.725e-06 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414563+0000", 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2530000000000001e-06 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414566+0000", 2024-09-16T11:45:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4429999999999999e-06 2024-09-16T11:45:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444463+0000", 2024-09-16T11:45:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029897271 2024-09-16T11:45:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444503+0000", 2024-09-16T11:45:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9782e-05 2024-09-16T11:45:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444552+0000", 2024-09-16T11:45:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8789000000000003e-05 2024-09-16T11:45:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.413 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.414 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.414 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:01.415 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:45:01.416 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:01.413+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:01.516 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:01.514+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:01.523 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:01.523 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:01.523 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:01.523 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:01.533 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:45:01.616 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:45:01.614+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:45:01.631 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:01.631 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:01.631 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:01.631 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:01.632 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:01.640 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:45:01.718 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:01.716+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:01.740 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:01.740 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:01.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:01.740 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:01.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.740 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:01.740 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:45:01.741 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.81744039599999, 2024-09-16T11:45:01.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:45:01.741 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.741 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:01.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:01.741 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:45:01.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.741 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:45:01.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:45:01.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:45:01.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:45:01.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:45:01.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:45:01.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:45:01.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:45:01.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:45:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:45:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:45:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:45:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:45:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:45:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:01.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:45:01.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:45:01.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:01.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:45:01.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:45:01.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:01.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:45:01.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:45:01.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:45:01.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:45:01.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:45:01.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:45:01.746 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.746 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.81739470399999, 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:45:01.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.748 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:45:01.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:45:01.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:45:01.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:45:01.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:45:01.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:45:01.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:45:01.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:45:01.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:45:01.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:45:01.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:45:01.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:45:01.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:45:01.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:45:01.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:01.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:45:01.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:45:01.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:01.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:45:01.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:45:01.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:45:01.753 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.753 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.753 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.753 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:01.753 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:45:01.753 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.81733777399999, 2024-09-16T11:45:01.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:45:01.753 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.753 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.753 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:45:01.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:45:01.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:45:01.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:45:01.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:45:01.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:45:01.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:45:01.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:45:01.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:45:01.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:45:01.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:45:01.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:45:01.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:45:01.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:01.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:45:01.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:45:01.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:01.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:45:01.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:45:01.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:01.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:45:01.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:45:01.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:45:01.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:45:01.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:45:01.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:45:01.760 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.760 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.760 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.760 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:01.760 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:01.760 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.817312406, 2024-09-16T11:45:01.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:45:01.760 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.760 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:01.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:01.761 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:45:01.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.761 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:01.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:01.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:01.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:45:01.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:45:01.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:45:01.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:45:01.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:45:01.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:45:01.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:45:01.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:45:01.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:45:01.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:45:01.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:45:01.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:01.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:45:01.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:45:01.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:01.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:45:01.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:45:01.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:01.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:45:01.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:45:01.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:45:01.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:45:01.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:45:01.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:45:01.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.767 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.767 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:01.767 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:01.768 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.81726043800001, 2024-09-16T11:45:01.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:45:01.768 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.768 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:01.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:01.768 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:45:01.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.768 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:01.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:01.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:01.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:45:01.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:45:01.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:45:01.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:45:01.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:45:01.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:45:01.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:45:01.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:45:01.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:45:01.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:45:01.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:45:01.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:01.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:45:01.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:45:01.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:01.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:45:01.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:45:01.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:01.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:45:01.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:45:01.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:45:01.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:45:01.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:45:01.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:45:01.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.774 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.774 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:01.774 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:01.774 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.81717932799999, 2024-09-16T11:45:01.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:45:01.774 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.775 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:01.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:01.775 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:45:01.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.775 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:01.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:01.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:01.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:45:01.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:45:01.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:45:01.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:45:01.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:45:01.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:45:01.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:45:01.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:45:01.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:45:01.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:45:01.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:45:01.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:01.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:45:01.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:45:01.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:01.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:45:01.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:45:01.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:01.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:45:01.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:45:01.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:45:01.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:45:01.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:45:01.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:45:01.781 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.781 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.781 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.781 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.81715335, 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:01.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:45:01.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:45:01.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:45:01.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:45:01.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:45:01.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:45:01.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:45:01.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:45:01.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:45:01.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.787 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.787 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:01.787 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:45:01.787 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.399634725, 2024-09-16T11:45:01.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:45:01.788 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.788 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.788 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.788 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:01.788 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:01.788 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:45:01.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.789 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:45:01.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:45:01.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:45:01.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:45:01.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:45:01.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:45:01.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:45:01.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:45:01.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:01.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:45:01.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:45:01.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:01.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:45:01.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:45:01.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:01.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:45:01.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:45:01.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:45:01.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:45:01.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:45:01.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:45:01.794 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.794 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.794 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.794 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:01.794 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:45:01.794 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.39961279400001, 2024-09-16T11:45:01.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:45:01.794 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.794 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:45:01.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:45:01.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:45:01.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:45:01.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:45:01.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:45:01.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:45:01.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:45:01.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:01.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:45:01.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:45:01.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:01.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:45:01.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:45:01.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:01.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:45:01.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:45:01.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:45:01.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:45:01.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:45:01.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:45:01.800 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.800 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.800 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.801 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:01.801 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:45:01.801 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.399535906, 2024-09-16T11:45:01.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:45:01.801 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.801 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.801 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:01.801 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:01.801 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:01.801 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:45:01.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.802 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:45:01.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:45:01.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:45:01.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:45:01.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:45:01.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:45:01.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:45:01.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:45:01.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:01.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:45:01.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:45:01.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:01.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:45:01.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:45:01.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:01.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:45:01.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:45:01.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:45:01.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:45:01.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:45:01.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:45:01.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.810 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.810 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28187 4.17 e52/28 4:ee916bd9:::rbd_data.11c1ce740b46.0000000000000000:head v 52'195, mlcod=52'193)", 2024-09-16T11:45:01.810 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:45:01.810 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.22811652, 2024-09-16T11:45:01.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075854429999999999, 2024-09-16T11:45:01.811 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.811 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.811 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:45:01.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:45:01.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509349+0000", 2024-09-16T11:45:01.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2300000000000001e-06 2024-09-16T11:45:01.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513315+0000", 2024-09-16T11:45:01.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039660590000000001 2024-09-16T11:45:01.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513328+0000", 2024-09-16T11:45:01.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.308e-05 2024-09-16T11:45:01.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513334+0000", 2024-09-16T11:45:01.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:45:01.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513385+0000", 2024-09-16T11:45:01.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1347000000000002e-05 2024-09-16T11:45:01.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513407+0000", 2024-09-16T11:45:01.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1852999999999999e-05 2024-09-16T11:45:01.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516882+0000", 2024-09-16T11:45:01.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034749690000000001 2024-09-16T11:45:01.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516933+0000", 2024-09-16T11:45:01.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0952999999999998e-05 2024-09-16T11:45:01.817 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.817 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.817 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.817 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28959 4.8 e53/28 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head v 53'408, mlcod=53'406)", 2024-09-16T11:45:01.817 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:45:01.818 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.705967463999997, 2024-09-16T11:45:01.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088740420000000004, 2024-09-16T11:45:01.818 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.818 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.818 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:45:01.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:45:01.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031498+0000", 2024-09-16T11:45:01.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.652e-06 2024-09-16T11:45:01.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033515+0000", 2024-09-16T11:45:01.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020165930000000001 2024-09-16T11:45:01.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033539+0000", 2024-09-16T11:45:01.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4295999999999999e-05 2024-09-16T11:45:01.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033548+0000", 2024-09-16T11:45:01.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1239999999999994e-06 2024-09-16T11:45:01.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033610+0000", 2024-09-16T11:45:01.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1371999999999994e-05 2024-09-16T11:45:01.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033637+0000", 2024-09-16T11:45:01.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7098999999999999e-05 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040341+0000", 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067045819999999997 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040371+0000", 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9323999999999999e-05 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.823 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32303 4.8 e53/28 4:1436ac82:::rbd_data.11c1cab081b3.0000000000000022:head v 53'417, mlcod=53'415)", 2024-09-16T11:45:01.823 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:01.823 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.235541490000003, 2024-09-16T11:45:01.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085565160000000001, 2024-09-16T11:45:01.823 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.823 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.823 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:01.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:01.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:01.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-09-16T11:45:01.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501930+0000", 2024-09-16T11:45:01.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7649999999999996e-06 2024-09-16T11:45:01.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501946+0000", 2024-09-16T11:45:01.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5301000000000001e-05 2024-09-16T11:45:01.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501952+0000", 2024-09-16T11:45:01.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:45:01.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502100+0000", 2024-09-16T11:45:01.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014847600000000001 2024-09-16T11:45:01.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502120+0000", 2024-09-16T11:45:01.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9303000000000001e-05 2024-09-16T11:45:01.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510436+0000", 2024-09-16T11:45:01.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083166420000000008 2024-09-16T11:45:01.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510479+0000", 2024-09-16T11:45:01.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2979000000000003e-05 2024-09-16T11:45:01.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.831 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.832 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32306 4.1e e53/28 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head v 53'124, mlcod=53'122)", 2024-09-16T11:45:01.832 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:01.832 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.235475495999999, 2024-09-16T11:45:01.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084990900000000008, 2024-09-16T11:45:01.832 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.832 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.832 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:01.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:01.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:01.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:45:01.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501994+0000", 2024-09-16T11:45:01.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:45:01.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502002+0000", 2024-09-16T11:45:01.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3839999999999997e-06 2024-09-16T11:45:01.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502007+0000", 2024-09-16T11:45:01.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6290000000000001e-06 2024-09-16T11:45:01.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502454+0000", 2024-09-16T11:45:01.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00044736800000000002 2024-09-16T11:45:01.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502471+0000", 2024-09-16T11:45:01.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6648999999999999e-05 2024-09-16T11:45:01.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510455+0000", 2024-09-16T11:45:01.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0079841909999999999 2024-09-16T11:45:01.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510488+0000", 2024-09-16T11:45:01.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2948999999999999e-05 2024-09-16T11:45:01.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.836 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.837 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32307 4.8 e53/28 4:11c1c37d:::rbd_data.11c1cab081b3.0000000000000026:head v 53'418, mlcod=53'415)", 2024-09-16T11:45:01.837 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:01.837 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.234720160999998, 2024-09-16T11:45:01.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00776212, 2024-09-16T11:45:01.837 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.837 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.837 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:01.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:01.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:01.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:45:01.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502750+0000", 2024-09-16T11:45:01.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:45:01.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502760+0000", 2024-09-16T11:45:01.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:45:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502767+0000", 2024-09-16T11:45:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4219999999999997e-06 2024-09-16T11:45:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502802+0000", 2024-09-16T11:45:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5089999999999998e-05 2024-09-16T11:45:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502816+0000", 2024-09-16T11:45:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4092e-05 2024-09-16T11:45:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510483+0000", 2024-09-16T11:45:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00766704 2024-09-16T11:45:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510506+0000", 2024-09-16T11:45:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3195999999999999e-05 2024-09-16T11:45:01.842 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33583 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'416, mlcod=53'411)", 2024-09-16T11:45:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:45:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.526761751999999, 2024-09-16T11:45:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078132459999999994, 2024-09-16T11:45:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.843 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:45:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:45:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210703+0000", 2024-09-16T11:45:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:45:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210706+0000", 2024-09-16T11:45:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-06 2024-09-16T11:45:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210712+0000", 2024-09-16T11:45:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:45:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210715+0000", 2024-09-16T11:45:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:45:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210736+0000", 2024-09-16T11:45:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0293999999999999e-05 2024-09-16T11:45:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210746+0000", 2024-09-16T11:45:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0178e-05 2024-09-16T11:45:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218477+0000", 2024-09-16T11:45:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007731501 2024-09-16T11:45:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218516+0000", 2024-09-16T11:45:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8148000000000001e-05 2024-09-16T11:45:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33590 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'419, mlcod=53'415)", 2024-09-16T11:45:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:45:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.515703610999999, 2024-09-16T11:45:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082499879999999998, 2024-09-16T11:45:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:45:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:45:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221761+0000", 2024-09-16T11:45:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-16T11:45:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221765+0000", 2024-09-16T11:45:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1590000000000003e-06 2024-09-16T11:45:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221774+0000", 2024-09-16T11:45:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7490000000000004e-06 2024-09-16T11:45:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221778+0000", 2024-09-16T11:45:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0419999999999996e-06 2024-09-16T11:45:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221800+0000", 2024-09-16T11:45:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1926999999999999e-05 2024-09-16T11:45:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221810+0000", 2024-09-16T11:45:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0079e-05 2024-09-16T11:45:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.229980+0000", 2024-09-16T11:45:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081698469999999992 2024-09-16T11:45:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230010+0000", 2024-09-16T11:45:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0746e-05 2024-09-16T11:45:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33591 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'420, mlcod=53'415)", 2024-09-16T11:45:01.853 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.515142126999997, 2024-09-16T11:45:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007702576, 2024-09-16T11:45:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222326+0000", 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222337+0000", 2024-09-16T11:45:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1008e-05 2024-09-16T11:45:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222342+0000", 2024-09-16T11:45:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0050000000000004e-06 2024-09-16T11:45:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222366+0000", 2024-09-16T11:45:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3696000000000001e-05 2024-09-16T11:45:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222385+0000", 2024-09-16T11:45:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-16T11:45:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230012+0000", 2024-09-16T11:45:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076270940000000001 2024-09-16T11:45:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230025+0000", 2024-09-16T11:45:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2112e-05 2024-09-16T11:45:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.858 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:55282 4.1e e73/28 4:785299e2:::rbd_data.11c11a49c2da.0000000000000001:head v 73'324, mlcod=73'322)", 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.498445607999997, 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071431890000000003, 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:45:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239020+0000", 2024-09-16T11:45:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.049e-06 2024-09-16T11:45:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244238+0000", 2024-09-16T11:45:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052183489999999997 2024-09-16T11:45:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244252+0000", 2024-09-16T11:45:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4246999999999999e-05 2024-09-16T11:45:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244275+0000", 2024-09-16T11:45:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3093999999999999e-05 2024-09-16T11:45:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244314+0000", 2024-09-16T11:45:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8495000000000003e-05 2024-09-16T11:45:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244330+0000", 2024-09-16T11:45:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5917e-05 2024-09-16T11:45:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246121+0000", 2024-09-16T11:45:01.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017909239999999999 2024-09-16T11:45:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246162+0000", 2024-09-16T11:45:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1114e-05 2024-09-16T11:45:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:61651 4.11 e83/28 4:8f4a51c5:::rbd_data.11c175f8cab7.0000000000000014:head v 83'694, mlcod=81'692)", 2024-09-16T11:45:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:45:01.863 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.124026060999999, 2024-09-16T11:45:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.009466482, 2024-09-16T11:45:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:45:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:01.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:45:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613439+0000", 2024-09-16T11:45:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8400000000000003e-07 2024-09-16T11:45:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618867+0000", 2024-09-16T11:45:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054276059999999998 2024-09-16T11:45:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618883+0000", 2024-09-16T11:45:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6265000000000001e-05 2024-09-16T11:45:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618890+0000", 2024-09-16T11:45:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2980000000000003e-06 2024-09-16T11:45:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618950+0000", 2024-09-16T11:45:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0025999999999998e-05 2024-09-16T11:45:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618971+0000", 2024-09-16T11:45:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0458999999999998e-05 2024-09-16T11:45:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622869+0000", 2024-09-16T11:45:01.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038983450000000001 2024-09-16T11:45:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622905+0000", 2024-09-16T11:45:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599000000000003e-05 2024-09-16T11:45:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:01.869 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:01.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:45:01.870 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:01.868+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:01.968 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:01.969 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:01.969 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:45:01.969 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:01.970 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:45:01.968+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:45:01.978 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:45:01.992 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:45:02.072 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:02.069+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:02.121 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:02.121 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:02.121 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:45:02.121 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:45:02.121 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:02.130 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:45:02.136 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:45:02.137 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:45:02.152 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:45:02.153 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:45:02.168 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:45:02.168 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:45:02.172 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:02.170+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:02.183 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:45:02.183 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:45:02.199 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:45:02.199 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:45:02.215 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:45:02.215 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:45:02.231 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:45:02.231 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:45:02.236 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:02.236 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:02.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:45:02.247 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:45:02.247 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:45:02.299 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:45:02.297+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:45:02.344 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:02.344 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:02.353 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:45:02.394 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:02.392+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:02.475 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:02.473+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:02.491 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:02.491 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:02.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:45:02.553 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:45:02.585 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:02.583+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:02.647 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:02.648 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:02.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:45:02.700 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:02.698+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:02.758 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:02.758 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:02.767 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:45:02.808 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:02.806+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:02.879 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:02.877+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:02.905 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:02.905 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:02.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:45:02.980 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:02.978+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:03.011 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:03.011 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:03.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:45:03.081 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:03.078+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:03.157 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:03.157 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:03.168 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:45:03.212 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:03.210+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:03.265 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:03.266 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:03.274 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:45:03.283 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:03.281+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:03.384 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:03.382+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:03.412 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:03.412 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:03.421 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:45:03.485 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:03.483+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:03.520 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T11:45:03.537 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:03.537 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:03.547 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:45:03.591 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:03.589+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:03.649 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:03.649 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:03.658 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:45:03.687 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:03.684+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:03.788 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:03.785+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:03.796 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:03.796 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:03.804 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:45:03.889 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:03.887+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:03.942 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:03.942 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:03.950 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:45:03.991 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:03.989+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:04.088 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:04.088 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:04.092 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:45:04.090+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:45:04.096 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:45:04.193 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:45:04.191+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:45:04.193 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:04.193 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:04.202 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:45:04.259 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:45:04.294 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:04.291+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:04.299 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:04.299 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:04.307 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:45:04.375 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:45:04.375 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:45:04.390 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:45:04.390 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:45:04.396 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:45:04.394+0000 7fcd2cdf8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33337) UID: 0 2024-09-16T11:45:04.405 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:45:04.405 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:45:04.420 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T11:45:04.420 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:45:04.435 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:45:04.435 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:45:04.445 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:04.446 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:04.450 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:45:04.450 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:45:04.454 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:45:04.466 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:45:04.466 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:45:04.481 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:45:04.481 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:45:04.522 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:45:04.522 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:45:04.522 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:45:04.523 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T11:45:04.523 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:45:04.533 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:04.531+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:04.556 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:04.557 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:04.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:04.557 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:04.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.557 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19282 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:04.557 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:45:04.557 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.080715639000005, 2024-09-16T11:45:04.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521151000000002, 2024-09-16T11:45:04.557 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.557 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19282 2024-09-16T11:45:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:45:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:45:04.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472838+0000", 2024-09-16T11:45:04.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:45:04.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472848+0000", 2024-09-16T11:45:04.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9799999999999993e-06 2024-09-16T11:45:04.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472858+0000", 2024-09-16T11:45:04.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9429999999999995e-06 2024-09-16T11:45:04.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472862+0000", 2024-09-16T11:45:04.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4460000000000003e-06 2024-09-16T11:45:04.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501277+0000", 2024-09-16T11:45:04.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028414900999999999 2024-09-16T11:45:04.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501322+0000", 2024-09-16T11:45:04.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4880999999999999e-05 2024-09-16T11:45:04.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501358+0000", 2024-09-16T11:45:04.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6245000000000003e-05 2024-09-16T11:45:04.561 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19286 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7979008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.080342707, 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521860999999999, 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19286 2024-09-16T11:45:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473211+0000", 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:45:04.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473214+0000", 2024-09-16T11:45:04.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1080000000000001e-06 2024-09-16T11:45:04.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473226+0000", 2024-09-16T11:45:04.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2441e-05 2024-09-16T11:45:04.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473234+0000", 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729999999999998e-06 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501592+0000", 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028358491999999999 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501671+0000", 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8846999999999999e-05 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501732+0000", 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1118000000000007e-05 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20619 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7610368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.992993538999997, 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028500152000000001, 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.567 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20619 2024-09-16T11:45:04.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.567 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:45:04.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560560+0000", 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9499999999999998e-07 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560561+0000", 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.071e-06 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560565+0000", 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560567+0000", 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588949+0000", 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381878999999999 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588989+0000", 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0649000000000003e-05 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589059+0000", 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0239000000000003e-05 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20620 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7716864~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:04.570 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:45:04.571 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.992984139000001, 2024-09-16T11:45:04.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028627621999999998, 2024-09-16T11:45:04.571 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.571 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.571 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20620 2024-09-16T11:45:04.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.571 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562377+0000", 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018081729999999999 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562380+0000", 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0469999999999998e-06 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562386+0000", 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2040000000000002e-06 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562390+0000", 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589092+0000", 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026701798999999998 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589139+0000", 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7320000000000001e-05 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589196+0000", 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7201999999999999e-05 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.574 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.575 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.575 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41954 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.575 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:45:04.575 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139278349000001, 2024-09-16T11:45:04.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028594441000000002, 2024-09-16T11:45:04.575 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.575 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.575 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41954 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414275+0000", 2024-09-16T11:45:04.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:45:04.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414277+0000", 2024-09-16T11:45:04.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:45:04.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414282+0000", 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7559999999999996e-06 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414286+0000", 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442778+0000", 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028492237 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442816+0000", 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8198000000000003e-05 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442869+0000", 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2589000000000001e-05 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41955 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.579 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139263290999999, 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028687865, 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41955 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:04.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414292+0000", 2024-09-16T11:45:04.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-09-16T11:45:04.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414299+0000", 2024-09-16T11:45:04.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4279999999999999e-06 2024-09-16T11:45:04.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414303+0000", 2024-09-16T11:45:04.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:45:04.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442892+0000", 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028589564000000001 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442930+0000", 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330999999999999e-05 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442977+0000", 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7874000000000003e-05 2024-09-16T11:45:04.583 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.584 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.584 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.584 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41956 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.584 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:45:04.584 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139227974000001, 2024-09-16T11:45:04.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028766896, 2024-09-16T11:45:04.584 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.584 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.585 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41956 2024-09-16T11:45:04.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.585 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:45:04.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:45:04.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414326+0000", 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9900000000000009e-07 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414330+0000", 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2339999999999996e-06 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414341+0000", 2024-09-16T11:45:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1358999999999999e-05 2024-09-16T11:45:04.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414347+0000", 2024-09-16T11:45:04.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8100000000000003e-06 2024-09-16T11:45:04.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443002+0000", 2024-09-16T11:45:04.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654331000000002 2024-09-16T11:45:04.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443040+0000", 2024-09-16T11:45:04.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8262000000000001e-05 2024-09-16T11:45:04.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443092+0000", 2024-09-16T11:45:04.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1901000000000003e-05 2024-09-16T11:45:04.588 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.588 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.588 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.589 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41957 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.589 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:45:04.589 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139200555000002, 2024-09-16T11:45:04.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028848318000000001, 2024-09-16T11:45:04.589 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.589 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41957 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414353+0000", 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414355+0000", 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3489999999999999e-06 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414364+0000", 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1449999999999997e-06 2024-09-16T11:45:04.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414368+0000", 2024-09-16T11:45:04.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8940000000000003e-06 2024-09-16T11:45:04.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443115+0000", 2024-09-16T11:45:04.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028746532000000002 2024-09-16T11:45:04.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443152+0000", 2024-09-16T11:45:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6814999999999999e-05 2024-09-16T11:45:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443201+0000", 2024-09-16T11:45:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9069000000000001e-05 2024-09-16T11:45:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.593 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41958 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.594 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:45:04.594 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139176773000003, 2024-09-16T11:45:04.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028939856, 2024-09-16T11:45:04.594 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.594 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.594 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41958 2024-09-16T11:45:04.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.594 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:45:04.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:45:04.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414377+0000", 2024-09-16T11:45:04.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4099999999999999e-07 2024-09-16T11:45:04.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414379+0000", 2024-09-16T11:45:04.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.418e-06 2024-09-16T11:45:04.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414385+0000", 2024-09-16T11:45:04.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999999e-06 2024-09-16T11:45:04.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414389+0000", 2024-09-16T11:45:04.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7740000000000002e-06 2024-09-16T11:45:04.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443223+0000", 2024-09-16T11:45:04.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028833919 2024-09-16T11:45:04.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443263+0000", 2024-09-16T11:45:04.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0148999999999998e-05 2024-09-16T11:45:04.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443316+0000", 2024-09-16T11:45:04.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2636000000000002e-05 2024-09-16T11:45:04.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.600 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.600 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41959 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.600 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:04.601 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139159847999998, 2024-09-16T11:45:04.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029040309, 2024-09-16T11:45:04.601 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.601 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.601 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41959 2024-09-16T11:45:04.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.601 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414395+0000", 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7579999999999999e-06 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414401+0000", 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5330000000000002e-06 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414404+0000", 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.529e-06 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443339+0000", 2024-09-16T11:45:04.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028934933999999999 2024-09-16T11:45:04.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443383+0000", 2024-09-16T11:45:04.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3832000000000003e-05 2024-09-16T11:45:04.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443433+0000", 2024-09-16T11:45:04.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0309000000000002e-05 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41960 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139145014, 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029138134, 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41960 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:04.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-07 2024-09-16T11:45:04.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414410+0000", 2024-09-16T11:45:04.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:45:04.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414417+0000", 2024-09-16T11:45:04.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:45:04.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414420+0000", 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443457+0000", 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029037065000000001 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443495+0000", 2024-09-16T11:45:04.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7555000000000002e-05 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443546+0000", 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1422000000000003e-05 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41961 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139129033000003, 2024-09-16T11:45:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029230973, 2024-09-16T11:45:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41961 2024-09-16T11:45:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4499999999999998e-07 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414432+0000", 2024-09-16T11:45:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.413e-06 2024-09-16T11:45:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414439+0000", 2024-09-16T11:45:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4769999999999997e-06 2024-09-16T11:45:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414443+0000", 2024-09-16T11:45:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8859999999999997e-06 2024-09-16T11:45:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443569+0000", 2024-09-16T11:45:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029126124 2024-09-16T11:45:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443606+0000", 2024-09-16T11:45:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6588999999999998e-05 2024-09-16T11:45:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443655+0000", 2024-09-16T11:45:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9138999999999998e-05 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41962 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139106065999997, 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029324505000000001, 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41962 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:04.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:45:04.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414449+0000", 2024-09-16T11:45:04.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0279999999999999e-06 2024-09-16T11:45:04.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414456+0000", 2024-09-16T11:45:04.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3339999999999998e-06 2024-09-16T11:45:04.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414459+0000", 2024-09-16T11:45:04.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.382e-06 2024-09-16T11:45:04.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443678+0000", 2024-09-16T11:45:04.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029218958999999999 2024-09-16T11:45:04.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443719+0000", 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0722000000000001e-05 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443771+0000", 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2661000000000003e-05 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41963 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.618 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139090162999999, 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029416557999999999, 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41963 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:04.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:04.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:04.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-16T11:45:04.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414465+0000", 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414470+0000", 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4750000000000001e-06 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414474+0000", 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3720000000000001e-06 2024-09-16T11:45:04.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443795+0000", 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029320821 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443830+0000", 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5463000000000002e-05 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443879+0000", 2024-09-16T11:45:04.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9147000000000002e-05 2024-09-16T11:45:04.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.623 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.623 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41964 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.623 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:04.623 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139075275000003, 2024-09-16T11:45:04.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029521202999999999, 2024-09-16T11:45:04.623 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.623 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.624 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41964 2024-09-16T11:45:04.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.624 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:04.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:04.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414480+0000", 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414486+0000", 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414489+0000", 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2770000000000001e-06 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443903+0000", 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029414109000000001 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443944+0000", 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0992999999999999e-05 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443999+0000", 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4632000000000001e-05 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41965 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139060313000002, 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029611293, 2024-09-16T11:45:04.627 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41965 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414495+0000", 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.88e-06 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414501+0000", 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2489999999999998e-06 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414504+0000", 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3139999999999999e-06 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444023+0000", 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029518564000000001 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444058+0000", 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5196e-05 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444104+0000", 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5732999999999997e-05 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.631 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41966 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139044106999997, 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029707678000000001, 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41966 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:04.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414514+0000", 2024-09-16T11:45:04.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-16T11:45:04.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414518+0000", 2024-09-16T11:45:04.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3839999999999999e-06 2024-09-16T11:45:04.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414521+0000", 2024-09-16T11:45:04.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0400000000000001e-06 2024-09-16T11:45:04.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444128+0000", 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029606612000000001 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444167+0000", 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9418000000000001e-05 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444216+0000", 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9354999999999998e-05 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.635 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41967 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139028297999999, 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029796916, 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41967 2024-09-16T11:45:04.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6699999999999999e-07 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414527+0000", 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7060000000000001e-06 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414533+0000", 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2140000000000001e-06 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414536+0000", 2024-09-16T11:45:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:45:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444239+0000", 2024-09-16T11:45:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029702698999999999 2024-09-16T11:45:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444273+0000", 2024-09-16T11:45:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4229e-05 2024-09-16T11:45:04.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444321+0000", 2024-09-16T11:45:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8477999999999999e-05 2024-09-16T11:45:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41968 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:45:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.139013401, 2024-09-16T11:45:04.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029902765000000001, 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41968 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414540+0000", 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414542+0000", 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:45:04.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414547+0000", 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4639999999999997e-06 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414551+0000", 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6279999999999998e-06 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444346+0000", 2024-09-16T11:45:04.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029794885 2024-09-16T11:45:04.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444389+0000", 2024-09-16T11:45:04.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3442000000000002e-05 2024-09-16T11:45:04.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444442+0000", 2024-09-16T11:45:04.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3173000000000001e-05 2024-09-16T11:45:04.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.644 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41969 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:45:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.138998489000002, 2024-09-16T11:45:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029997612999999999, 2024-09-16T11:45:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41969 2024-09-16T11:45:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:04.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414555+0000", 2024-09-16T11:45:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:45:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414556+0000", 2024-09-16T11:45:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.725e-06 2024-09-16T11:45:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414563+0000", 2024-09-16T11:45:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2530000000000001e-06 2024-09-16T11:45:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:04.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414566+0000", 2024-09-16T11:45:04.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4429999999999999e-06 2024-09-16T11:45:04.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:04.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444463+0000", 2024-09-16T11:45:04.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029897271 2024-09-16T11:45:04.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:04.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444503+0000", 2024-09-16T11:45:04.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9782e-05 2024-09-16T11:45:04.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:04.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:04.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:04.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444552+0000", 2024-09-16T11:45:04.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8789000000000003e-05 2024-09-16T11:45:04.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.649 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:04.649 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:04.649 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:04.651 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:04.649+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:04.652 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:45:04.752 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:04.749+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:04.790 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:04.790 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:04.790 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:45:04.790 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:04.799 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:45:04.852 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:04.850+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:04.897 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:04.897 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:04.897 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:45:04.897 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:45:04.897 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:04.906 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:45:04.953 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:04.952+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:05.045 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:05.045 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:45:05.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:45:05.045 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89607 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1327104~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.115235527999999, 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.033363680999999999, 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89607 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929100+0000", 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001870776 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929104+0000", 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1160000000000001e-06 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929118+0000", 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3891e-05 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929126+0000", 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6590000000000007e-06 2024-09-16T11:45:05.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.049 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960508+0000", 2024-09-16T11:45:05.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031381920000000001 2024-09-16T11:45:05.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.049 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960548+0000", 2024-09-16T11:45:05.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0734e-05 2024-09-16T11:45:05.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960593+0000", 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4585000000000001e-05 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89611 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1687552~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.113253568000001, 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031799929999999997, 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.050 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89611 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:45:05.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929212+0000", 2024-09-16T11:45:05.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0800000000000001e-07 2024-09-16T11:45:05.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929213+0000", 2024-09-16T11:45:05.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6509999999999999e-06 2024-09-16T11:45:05.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929218+0000", 2024-09-16T11:45:05.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9899999999999997e-06 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929221+0000", 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1140000000000002e-06 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960926+0000", 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031704436000000003 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960966+0000", 2024-09-16T11:45:05.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0055999999999997e-05 2024-09-16T11:45:05.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.055 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961011+0000", 2024-09-16T11:45:05.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5374999999999998e-05 2024-09-16T11:45:05.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.055 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.056 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89612 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1777664~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.056 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:45:05.056 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.113240358999999, 2024-09-16T11:45:05.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031893013999999997, 2024-09-16T11:45:05.056 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.056 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.056 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.056 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.056 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.056 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89612 2024-09-16T11:45:05.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.056 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:45:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:45:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6e-07 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929226+0000", 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5099999999999999e-06 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929231+0000", 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4889999999999997e-06 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929234+0000", 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7410000000000001e-06 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961033+0000", 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031799279 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961071+0000", 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7759000000000002e-05 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961118+0000", 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6975999999999999e-05 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89613 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1867776~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.11322822, 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031996359000000002, 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.060 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.061 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.061 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.061 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89613 2024-09-16T11:45:05.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.061 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:45:05.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8999999999999998e-07 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929243+0000", 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8719999999999999e-06 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929249+0000", 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6229999999999999e-06 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929253+0000", 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961145+0000", 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031892648000000003 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.064 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961188+0000", 2024-09-16T11:45:05.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2466999999999998e-05 2024-09-16T11:45:05.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.064 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961233+0000", 2024-09-16T11:45:05.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5049999999999997e-05 2024-09-16T11:45:05.064 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.064 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.064 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.065 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89615 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1957888~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.065 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:45:05.065 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.113208350000001, 2024-09-16T11:45:05.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032076532999999997, 2024-09-16T11:45:05.065 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.065 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.065 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.065 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.065 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.065 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89615 2024-09-16T11:45:05.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:45:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:45:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.067 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:45:05.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:45:05.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.067 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.067 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929259+0000", 2024-09-16T11:45:05.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7269999999999999e-06 2024-09-16T11:45:05.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.067 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.067 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929264+0000", 2024-09-16T11:45:05.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.524e-06 2024-09-16T11:45:05.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929267+0000", 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.994e-06 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961252+0000", 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031985188999999997 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961291+0000", 2024-09-16T11:45:05.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8263000000000003e-05 2024-09-16T11:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961333+0000", 2024-09-16T11:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2491999999999999e-05 2024-09-16T11:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89617 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 2048000~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.069 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:45:05.070 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.113194716999999, 2024-09-16T11:45:05.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032185244000000002, 2024-09-16T11:45:05.070 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.070 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.070 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.070 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.070 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.070 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89617 2024-09-16T11:45:05.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.070 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0499999999999999e-07 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929272+0000", 2024-09-16T11:45:05.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.615e-06 2024-09-16T11:45:05.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929279+0000", 2024-09-16T11:45:05.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5459999999999997e-06 2024-09-16T11:45:05.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929282+0000", 2024-09-16T11:45:05.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.472e-06 2024-09-16T11:45:05.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961361+0000", 2024-09-16T11:45:05.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032078441999999999 2024-09-16T11:45:05.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961401+0000", 2024-09-16T11:45:05.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0425e-05 2024-09-16T11:45:05.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961455+0000", 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4438999999999998e-05 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89637 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4235264~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.11317446, 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032120463000000002, 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.074 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.075 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.075 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.075 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89637 2024-09-16T11:45:05.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.075 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:45:05.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929291+0000", 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929292+0000", 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3120000000000001e-06 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.077 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929295+0000", 2024-09-16T11:45:05.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3349999999999998e-06 2024-09-16T11:45:05.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.077 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929297+0000", 2024-09-16T11:45:05.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0549999999999998e-06 2024-09-16T11:45:05.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.077 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961346+0000", 2024-09-16T11:45:05.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032048920000000002 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961379+0000", 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2533000000000002e-05 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961411+0000", 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1921000000000002e-05 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.079 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89639 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4325376~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.079 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:45:05.079 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.113165381000002, 2024-09-16T11:45:05.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032194749000000002, 2024-09-16T11:45:05.079 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.079 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.080 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.080 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.080 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.080 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89639 2024-09-16T11:45:05.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.081 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:45:05.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:45:05.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929300+0000", 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929301+0000", 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.018e-06 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929304+0000", 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8890000000000002e-06 2024-09-16T11:45:05.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929306+0000", 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.883e-06 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961422+0000", 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032116899999999997 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961460+0000", 2024-09-16T11:45:05.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7265999999999999e-05 2024-09-16T11:45:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961494+0000", 2024-09-16T11:45:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4499000000000003e-05 2024-09-16T11:45:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89641 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4415488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:45:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.113157458, 2024-09-16T11:45:05.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032287719999999999, 2024-09-16T11:45:05.085 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.085 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.085 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.085 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.085 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.085 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89641 2024-09-16T11:45:05.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.085 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:45:05.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929308+0000", 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1500000000000001e-07 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929309+0000", 2024-09-16T11:45:05.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.02e-06 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929312+0000", 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3000000000000002e-06 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929314+0000", 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0449999999999999e-06 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961521+0000", 2024-09-16T11:45:05.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032206835000000003 2024-09-16T11:45:05.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961561+0000", 2024-09-16T11:45:05.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0580000000000001e-05 2024-09-16T11:45:05.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961595+0000", 2024-09-16T11:45:05.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3725e-05 2024-09-16T11:45:05.088 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.088 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.088 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.089 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89643 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4505600~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.089 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:45:05.089 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.113149020000002, 2024-09-16T11:45:05.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032377892999999998, 2024-09-16T11:45:05.089 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.089 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.089 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.090 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.090 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.090 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89643 2024-09-16T11:45:05.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.090 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:45:05.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:45:05.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:45:05.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0100000000000001e-07 2024-09-16T11:45:05.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929317+0000", 2024-09-16T11:45:05.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.161e-06 2024-09-16T11:45:05.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929320+0000", 2024-09-16T11:45:05.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1580000000000001e-06 2024-09-16T11:45:05.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929322+0000", 2024-09-16T11:45:05.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9929999999999998e-06 2024-09-16T11:45:05.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961619+0000", 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032296392 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961659+0000", 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9805000000000003e-05 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961694+0000", 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5182999999999998e-05 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.093 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.094 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.094 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89644 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4243456~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.094 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:45:05.094 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.113140641000001, 2024-09-16T11:45:05.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032467912000000002, 2024-09-16T11:45:05.094 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.094 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.094 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.094 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.094 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.094 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89644 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0100000000000001e-07 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929325+0000", 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9899999999999999e-07 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929328+0000", 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7539999999999998e-06 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.097 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.097 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929330+0000", 2024-09-16T11:45:05.097 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3690000000000001e-06 2024-09-16T11:45:05.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.097 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.097 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961725+0000", 2024-09-16T11:45:05.097 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032394088000000001 2024-09-16T11:45:05.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.097 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.098 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961759+0000", 2024-09-16T11:45:05.098 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4316999999999997e-05 2024-09-16T11:45:05.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.098 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.098 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961792+0000", 2024-09-16T11:45:05.099 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3284000000000002e-05 2024-09-16T11:45:05.099 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.099 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.099 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.099 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89645 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4595712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.100 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:45:05.100 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.113132569000001, 2024-09-16T11:45:05.100 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032554313000000001, 2024-09-16T11:45:05.100 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.100 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.100 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.100 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.101 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.101 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89645 2024-09-16T11:45:05.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.101 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.101 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.101 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:45:05.102 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.102 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.102 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:45:05.102 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.103 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:45:05.103 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.99e-07 2024-09-16T11:45:05.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.103 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929333+0000", 2024-09-16T11:45:05.104 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9299999999999996e-07 2024-09-16T11:45:05.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.104 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.104 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929336+0000", 2024-09-16T11:45:05.104 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8459999999999999e-06 2024-09-16T11:45:05.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929340+0000", 2024-09-16T11:45:05.105 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6370000000000001e-06 2024-09-16T11:45:05.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.106 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.106 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961814+0000", 2024-09-16T11:45:05.106 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032474461000000003 2024-09-16T11:45:05.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.106 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.106 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961852+0000", 2024-09-16T11:45:05.106 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7877999999999999e-05 2024-09-16T11:45:05.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.106 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.106 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961887+0000", 2024-09-16T11:45:05.107 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4399e-05 2024-09-16T11:45:05.107 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.107 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.107 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.107 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.107 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89646 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4333568~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.107 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:45:05.107 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.113122932, 2024-09-16T11:45:05.107 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032641146000000003, 2024-09-16T11:45:05.107 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.107 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.107 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.108 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.108 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.108 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89646 2024-09-16T11:45:05.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.108 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.108 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.108 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:45:05.108 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.109 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.109 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:45:05.109 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.109 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.109 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:45:05.109 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.04e-07 2024-09-16T11:45:05.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.109 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929343+0000", 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7000000000000003e-07 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929346+0000", 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7350000000000001e-06 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929349+0000", 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7030000000000002e-06 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.111 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.111 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961916+0000", 2024-09-16T11:45:05.111 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032567634999999998 2024-09-16T11:45:05.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.111 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.111 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961950+0000", 2024-09-16T11:45:05.111 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3510000000000003e-05 2024-09-16T11:45:05.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.111 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.111 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961983+0000", 2024-09-16T11:45:05.111 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3389000000000002e-05 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89647 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4685824~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.113114530000001, 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032727342, 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.112 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89647 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931111+0000", 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001760462 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931114+0000", 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.67e-06 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931122+0000", 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6059999999999997e-06 2024-09-16T11:45:05.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.115 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.115 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931126+0000", 2024-09-16T11:45:05.115 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-06 2024-09-16T11:45:05.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.115 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.115 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962005+0000", 2024-09-16T11:45:05.115 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.030879125 2024-09-16T11:45:05.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.115 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.115 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962044+0000", 2024-09-16T11:45:05.116 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8779000000000003e-05 2024-09-16T11:45:05.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.116 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.116 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.116 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962078+0000", 2024-09-16T11:45:05.116 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4010000000000001e-05 2024-09-16T11:45:05.116 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.116 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.116 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.116 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.116 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89656 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4784128~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.116 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:45:05.117 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.111264480999999, 2024-09-16T11:45:05.117 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031742976999999999, 2024-09-16T11:45:05.117 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.117 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.117 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.117 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.117 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.117 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89656 2024-09-16T11:45:05.117 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.117 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.117 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.117 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.117 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931201+0000", 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0699999999999999e-07 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.119 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931202+0000", 2024-09-16T11:45:05.119 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5900000000000005e-07 2024-09-16T11:45:05.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.119 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.119 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931204+0000", 2024-09-16T11:45:05.119 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8439999999999998e-06 2024-09-16T11:45:05.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.119 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.119 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931206+0000", 2024-09-16T11:45:05.119 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.871e-06 2024-09-16T11:45:05.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.120 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.120 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962877+0000", 2024-09-16T11:45:05.120 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031670831000000003 2024-09-16T11:45:05.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.120 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.120 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962910+0000", 2024-09-16T11:45:05.120 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3405999999999998e-05 2024-09-16T11:45:05.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.121 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.121 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962943+0000", 2024-09-16T11:45:05.121 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2858999999999998e-05 2024-09-16T11:45:05.121 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.121 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.121 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.121 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89657 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5136384~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.121 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:45:05.122 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.11125667, 2024-09-16T11:45:05.122 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031827938, 2024-09-16T11:45:05.122 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.122 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.122 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.122 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.122 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.122 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89657 2024-09-16T11:45:05.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.123 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.123 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.123 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:45:05.123 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.123 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.123 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.123 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:45:05.123 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.124 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.124 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.124 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:45:05.124 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.28e-07 2024-09-16T11:45:05.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.124 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.124 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.124 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931209+0000", 2024-09-16T11:45:05.124 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9099999999999991e-07 2024-09-16T11:45:05.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.125 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.125 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931213+0000", 2024-09-16T11:45:05.125 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7639999999999999e-06 2024-09-16T11:45:05.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.125 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.125 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931217+0000", 2024-09-16T11:45:05.125 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4759999999999998e-06 2024-09-16T11:45:05.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.125 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962964+0000", 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031747598000000002 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963003+0000", 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8287000000000001e-05 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963036+0000", 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3593999999999997e-05 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.126 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.127 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.127 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.127 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.127 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89658 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4874240~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.127 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:45:05.127 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.111229174999998, 2024-09-16T11:45:05.127 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031905067000000002, 2024-09-16T11:45:05.127 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.127 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.127 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.127 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.127 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.127 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89658 2024-09-16T11:45:05.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.128 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.128 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.128 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:45:05.128 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.128 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.128 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:45:05.128 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4700000000000001e-07 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931238+0000", 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931245+0000", 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.64e-06 2024-09-16T11:45:05.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931248+0000", 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5719999999999999e-06 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963066+0000", 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031817326999999999 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963107+0000", 2024-09-16T11:45:05.130 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1220000000000002e-05 2024-09-16T11:45:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963141+0000", 2024-09-16T11:45:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3896999999999997e-05 2024-09-16T11:45:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89659 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4964352~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:45:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.111212872999999, 2024-09-16T11:45:05.132 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031984460999999999, 2024-09-16T11:45:05.132 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.132 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.132 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.132 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.132 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.132 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89659 2024-09-16T11:45:05.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.132 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.133 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.133 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:45:05.133 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.133 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.133 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:45:05.133 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.133 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.133 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931254+0000", 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.776e-06 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931262+0000", 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1680000000000004e-06 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.134 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931266+0000", 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.732e-06 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963165+0000", 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031898693999999998 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963202+0000", 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7517e-05 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.136 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963236+0000", 2024-09-16T11:45:05.136 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4230000000000003e-05 2024-09-16T11:45:05.136 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.136 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.136 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.136 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89660 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5226496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.136 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.111194853000001, 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032062368000000001, 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89660 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.39e-07 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931272+0000", 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:45:05.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931279+0000", 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931282+0000", 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.721e-06 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963266+0000", 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031983971999999999 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963300+0000", 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3318000000000002e-05 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963332+0000", 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2768000000000002e-05 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.140 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89661 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5054464~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:05.141 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:45:05.141 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.111178428999999, 2024-09-16T11:45:05.141 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032139394000000002, 2024-09-16T11:45:05.141 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.141 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.141 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.141 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.141 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.141 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89661 2024-09-16T11:45:05.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.141 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.141 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933104+0000", 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00181734 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933107+0000", 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5140000000000002e-06 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933115+0000", 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933124+0000", 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8910000000000001e-06 2024-09-16T11:45:05.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963354+0000", 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.030230825999999999 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963392+0000", 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7765000000000002e-05 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963426+0000", 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3603000000000003e-05 2024-09-16T11:45:05.144 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.145 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.145 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.145 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.145 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.145 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:05.146 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:45:05.147 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:05.144+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:05.245 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:05.245 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:05.245 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:45:05.245 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:05.246 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:05.244+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:05.253 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:45:05.348 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:05.345+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:05.350 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:05.350 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:05.350 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:45:05.351 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:45:05.351 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:05.359 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:45:05.449 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:05.446+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:05.499 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:05.499 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:45:05.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:45:05.499 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:45:05.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.499 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19364 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1171456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.499 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:45:05.499 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.983094855999994, 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049001805000000002, 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19364 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513280+0000", 2024-09-16T11:45:05.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7090000000000001e-06 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513286+0000", 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5840000000000002e-06 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513290+0000", 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3680000000000004e-06 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562121+0000", 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048831324000000002 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562195+0000", 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3415000000000001e-05 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562280+0000", 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4975999999999994e-05 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.503 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.504 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19365 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1490944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.504 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:45:05.504 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.983075116999998, 2024-09-16T11:45:05.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049335349000000001, 2024-09-16T11:45:05.504 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.504 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.504 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.504 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.504 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.504 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19365 2024-09-16T11:45:05.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513300+0000", 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3020000000000002e-06 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513306+0000", 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8780000000000003e-06 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513310+0000", 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1309999999999999e-06 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562322+0000", 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049011635999999997 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562544+0000", 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022168000000000001 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562633+0000", 2024-09-16T11:45:05.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9350000000000003e-05 2024-09-16T11:45:05.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.508 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.508 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19366 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1916928~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.508 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:45:05.508 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.983058185000004, 2024-09-16T11:45:05.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049865073000000003, 2024-09-16T11:45:05.508 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19366 2024-09-16T11:45:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:45:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:45:05.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515102+0000", 2024-09-16T11:45:05.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017877419999999999 2024-09-16T11:45:05.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515106+0000", 2024-09-16T11:45:05.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4450000000000001e-06 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515113+0000", 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7680000000000008e-06 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515117+0000", 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562756+0000", 2024-09-16T11:45:05.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047638579 2024-09-16T11:45:05.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563093+0000", 2024-09-16T11:45:05.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000337516 2024-09-16T11:45:05.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563180+0000", 2024-09-16T11:45:05.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6420000000000003e-05 2024-09-16T11:45:05.512 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.513 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.513 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.513 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19371 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3194880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.513 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:45:05.513 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.981160978000005, 2024-09-16T11:45:05.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048948356999999998, 2024-09-16T11:45:05.513 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.513 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.513 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.513 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.513 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19371 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:45:05.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:45:05.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515214+0000", 2024-09-16T11:45:05.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5969999999999999e-06 2024-09-16T11:45:05.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515219+0000", 2024-09-16T11:45:05.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4559999999999999e-06 2024-09-16T11:45:05.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515223+0000", 2024-09-16T11:45:05.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:45:05.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563991+0000", 2024-09-16T11:45:05.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048768911999999998 2024-09-16T11:45:05.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564063+0000", 2024-09-16T11:45:05.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2010999999999999e-05 2024-09-16T11:45:05.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564160+0000", 2024-09-16T11:45:05.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6617e-05 2024-09-16T11:45:05.517 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.517 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.517 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19372 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3620864~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.981146761000005, 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049130502, 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19372 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:45:05.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:45:05.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:45:05.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:45:05.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515228+0000", 2024-09-16T11:45:05.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T11:45:05.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515233+0000", 2024-09-16T11:45:05.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:45:05.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515237+0000", 2024-09-16T11:45:05.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.27e-06 2024-09-16T11:45:05.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564199+0000", 2024-09-16T11:45:05.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048961787999999999 2024-09-16T11:45:05.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564275+0000", 2024-09-16T11:45:05.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6001000000000006e-05 2024-09-16T11:45:05.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564356+0000", 2024-09-16T11:45:05.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1941999999999996e-05 2024-09-16T11:45:05.522 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.522 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.522 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.522 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19373 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3940352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.522 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:45:05.522 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.981132360999993, 2024-09-16T11:45:05.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049289588000000002, 2024-09-16T11:45:05.523 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.523 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.523 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.523 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.523 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.523 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19373 2024-09-16T11:45:05.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.523 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.523 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:45:05.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:45:05.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515241+0000", 2024-09-16T11:45:05.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:45:05.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515242+0000", 2024-09-16T11:45:05.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.528e-06 2024-09-16T11:45:05.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.525 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515248+0000", 2024-09-16T11:45:05.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:45:05.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.525 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515251+0000", 2024-09-16T11:45:05.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2480000000000001e-06 2024-09-16T11:45:05.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564388+0000", 2024-09-16T11:45:05.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049137313000000002 2024-09-16T11:45:05.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564455+0000", 2024-09-16T11:45:05.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7049000000000001e-05 2024-09-16T11:45:05.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564530+0000", 2024-09-16T11:45:05.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4626e-05 2024-09-16T11:45:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19374 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 4046848~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:45:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.981117957999999, 2024-09-16T11:45:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049457133, 2024-09-16T11:45:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.528 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.528 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19374 2024-09-16T11:45:05.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.528 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.528 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.528 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:45:05.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.528 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.528 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:45:05.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.529 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.529 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:45:05.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:45:05.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.529 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.529 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515257+0000", 2024-09-16T11:45:05.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5740000000000001e-06 2024-09-16T11:45:05.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.529 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515262+0000", 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2399999999999998e-06 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515265+0000", 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3909999999999998e-06 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564555+0000", 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049290002999999999 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564628+0000", 2024-09-16T11:45:05.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2559999999999996e-05 2024-09-16T11:45:05.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564712+0000", 2024-09-16T11:45:05.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4040000000000002e-05 2024-09-16T11:45:05.531 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.531 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.531 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.532 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19375 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3301376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.532 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:45:05.532 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.981103814999997, 2024-09-16T11:45:05.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049606631999999998, 2024-09-16T11:45:05.532 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.532 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.532 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.532 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.532 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.532 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19375 2024-09-16T11:45:05.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.532 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515271+0000", 2024-09-16T11:45:05.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5230000000000001e-06 2024-09-16T11:45:05.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515276+0000", 2024-09-16T11:45:05.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4650000000000002e-06 2024-09-16T11:45:05.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515279+0000", 2024-09-16T11:45:05.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0630000000000002e-06 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564738+0000", 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049459001000000002 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564780+0000", 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1693999999999998e-05 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564876+0000", 2024-09-16T11:45:05.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5571999999999999e-05 2024-09-16T11:45:05.536 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.536 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.536 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.536 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19376 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3727360~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.536 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:45:05.536 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.981089902999997, 2024-09-16T11:45:05.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049720590000000002, 2024-09-16T11:45:05.536 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.536 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.537 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.537 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.537 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.537 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19376 2024-09-16T11:45:05.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.537 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:45:05.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:45:05.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517103+0000", 2024-09-16T11:45:05.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001820276 2024-09-16T11:45:05.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517108+0000", 2024-09-16T11:45:05.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5410000000000002e-06 2024-09-16T11:45:05.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517120+0000", 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2092e-05 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517125+0000", 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6829999999999996e-06 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564870+0000", 2024-09-16T11:45:05.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047745040000000002 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564921+0000", 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0538999999999998e-05 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565003+0000", 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2418999999999998e-05 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.541 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19492 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 434176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.541 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:45:05.541 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.979161047000005, 2024-09-16T11:45:05.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048926948999999997, 2024-09-16T11:45:05.541 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.541 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.541 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.541 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.541 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.541 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19492 2024-09-16T11:45:05.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.541 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517212+0000", 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3099999999999999e-07 2024-09-16T11:45:05.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517213+0000", 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6599999999999994e-07 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517216+0000", 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.706e-06 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517218+0000", 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.035e-06 2024-09-16T11:45:05.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565959+0000", 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048741380000000001 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566036+0000", 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7362999999999998e-05 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566139+0000", 2024-09-16T11:45:05.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102268 2024-09-16T11:45:05.545 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.545 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.545 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.545 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19493 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 860160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.545 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:45:05.545 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.979139376000006, 2024-09-16T11:45:05.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049086604999999998, 2024-09-16T11:45:05.545 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.545 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.545 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.545 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19493 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517234+0000", 2024-09-16T11:45:05.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.37e-07 2024-09-16T11:45:05.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517236+0000", 2024-09-16T11:45:05.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7150000000000001e-06 2024-09-16T11:45:05.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517243+0000", 2024-09-16T11:45:05.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9489999999999999e-06 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517246+0000", 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.636e-06 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566163+0000", 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048917284999999998 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566237+0000", 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3196999999999997e-05 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566320+0000", 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3286000000000002e-05 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19494 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1179648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:45:05.549 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.979118760000006, 2024-09-16T11:45:05.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049259562999999999, 2024-09-16T11:45:05.550 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.550 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.550 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.550 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.550 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.550 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19494 2024-09-16T11:45:05.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.550 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:45:05.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:45:05.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:45:05.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:45:05.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517256+0000", 2024-09-16T11:45:05.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.888e-06 2024-09-16T11:45:05.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517265+0000", 2024-09-16T11:45:05.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8629999999999997e-06 2024-09-16T11:45:05.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517269+0000", 2024-09-16T11:45:05.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2039999999999999e-06 2024-09-16T11:45:05.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566352+0000", 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049083107000000001 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566429+0000", 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7025000000000002e-05 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566514+0000", 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4121000000000003e-05 2024-09-16T11:45:05.553 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.554 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.554 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.554 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19495 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1286144~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.554 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:45:05.554 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.979099336999994, 2024-09-16T11:45:05.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049492880000000003, 2024-09-16T11:45:05.554 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.554 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.554 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.554 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.554 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19495 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.556 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.556 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517274+0000", 2024-09-16T11:45:05.556 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:45:05.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.556 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.556 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517276+0000", 2024-09-16T11:45:05.556 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8950000000000001e-06 2024-09-16T11:45:05.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.556 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.556 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517281+0000", 2024-09-16T11:45:05.556 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3879999999999999e-06 2024-09-16T11:45:05.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517285+0000", 2024-09-16T11:45:05.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5930000000000002e-06 2024-09-16T11:45:05.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566579+0000", 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049294175000000003 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566670+0000", 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1096999999999999e-05 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566766+0000", 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6355999999999997e-05 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.558 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19496 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1605632~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.979084252000007, 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049632981999999999, 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19496 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.560 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:45:05.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.560 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:45:05.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.560 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517289+0000", 2024-09-16T11:45:05.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:45:05.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.561 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517290+0000", 2024-09-16T11:45:05.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.581e-06 2024-09-16T11:45:05.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.561 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517300+0000", 2024-09-16T11:45:05.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:45:05.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517304+0000", 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566789+0000", 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049485102000000003 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566843+0000", 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3937000000000002e-05 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566921+0000", 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8397000000000002e-05 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19497 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 114688~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.979064557000001, 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049761367000000001, 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.563 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.564 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.564 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.564 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19497 2024-09-16T11:45:05.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.564 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:45:05.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519116+0000", 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018079439999999999 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519120+0000", 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2760000000000002e-06 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519134+0000", 2024-09-16T11:45:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3468999999999999e-05 2024-09-16T11:45:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519140+0000", 2024-09-16T11:45:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.516e-06 2024-09-16T11:45:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566943+0000", 2024-09-16T11:45:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047802561 2024-09-16T11:45:05.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566982+0000", 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9269e-05 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567070+0000", 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7331999999999997e-05 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19504 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 966656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.97713607, 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049045863000000002, 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19504 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:45:05.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:45:05.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:45:05.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:45:05.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519239+0000", 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.962e-06 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519244+0000", 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5480000000000001e-06 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519248+0000", 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.641e-06 2024-09-16T11:45:05.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568150+0000", 2024-09-16T11:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048901864000000003 2024-09-16T11:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568208+0000", 2024-09-16T11:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8399000000000001e-05 2024-09-16T11:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568283+0000", 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4105999999999998e-05 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19505 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1392640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.977120764999995, 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049202304000000002, 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.572 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19505 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:45:05.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:45:05.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519254+0000", 2024-09-16T11:45:05.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.761e-06 2024-09-16T11:45:05.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519257+0000", 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.343e-06 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519259+0000", 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0559999999999999e-06 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568307+0000", 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049047445000000002 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568382+0000", 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5574999999999994e-05 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568454+0000", 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1799999999999997e-05 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19506 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1712128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.576 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.977110726999996, 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049341532, 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19506 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:45:05.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521102+0000", 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018404700000000001 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521106+0000", 2024-09-16T11:45:05.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8820000000000003e-06 2024-09-16T11:45:05.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521119+0000", 2024-09-16T11:45:05.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2546e-05 2024-09-16T11:45:05.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521125+0000", 2024-09-16T11:45:05.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0249999999999999e-06 2024-09-16T11:45:05.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568484+0000", 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047359120999999997 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568542+0000", 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8360999999999999e-05 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568604+0000", 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1126999999999999e-05 2024-09-16T11:45:05.580 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19515 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1499136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 69.975118441999996, 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048949431000000002, 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.581 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19515 2024-09-16T11:45:05.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.582 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:45:05.582 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:45:05.582 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523130+0000", 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00187604 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523134+0000", 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523144+0000", 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2010000000000005e-06 2024-09-16T11:45:05.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523149+0000", 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0039999999999999e-06 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570113+0000", 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.046964517999999997 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570157+0000", 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4056e-05 2024-09-16T11:45:05.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.585 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570204+0000", 2024-09-16T11:45:05.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6591999999999997e-05 2024-09-16T11:45:05.585 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.585 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.585 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.585 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:44111 4.8 4:17f3f47e:::rbd_data.11c1e170f5f0.000000000000002f:head [delete] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:45:05.585 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:45:05.585 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 55.513138161000001, 2024-09-16T11:45:05.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93903426400000001, 2024-09-16T11:45:05.586 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.586 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:05.586 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.586 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.586 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.586 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 44111 2024-09-16T11:45:05.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.586 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.586 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:45:05.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.587 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:45:05.587 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.587 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:45:05.587 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7199999999999998e-07 2024-09-16T11:45:05.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983239+0000", 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9879999999999998e-06 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983248+0000", 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.371e-06 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983253+0000", 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1150000000000002e-06 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983303+0000", 2024-09-16T11:45:05.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0380000000000002e-05 2024-09-16T11:45:05.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:45:05.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983311+0000", 2024-09-16T11:45:05.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7519999999999996e-06 2024-09-16T11:45:05.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920618+0000", 2024-09-16T11:45:05.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93730672000000004 2024-09-16T11:45:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920730+0000", 2024-09-16T11:45:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011265900000000001 2024-09-16T11:45:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:45:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920842+0000", 2024-09-16T11:45:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111638 2024-09-16T11:45:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.921503+0000", 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066074299999999999 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922208+0000", 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00070492899999999997 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922233+0000", 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5922999999999999e-05 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.592 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.592 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922269+0000", 2024-09-16T11:45:05.592 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5373999999999997e-05 2024-09-16T11:45:05.592 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.592 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.592 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.592 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.592 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.592 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:05.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:45:05.594 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:45:05.595 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:05.592+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:05.694 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:05.692+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:05.735 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:05.735 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:05.735 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:45:05.735 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:05.745 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:45:05.796 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:05.794+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:05.883 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:05.883 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:05.883 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:45:05.883 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:45:05.883 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:05.891 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:45:05.896 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:05.894+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:05.991 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:05.991 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:45:05.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:45:05.992 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:45:05.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.992 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16853 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3375104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.992 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:45:05.992 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.641411523000002, 2024-09-16T11:45:05.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027791907000000001, 2024-09-16T11:45:05.992 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.992 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.992 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.992 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.992 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.993 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16853 2024-09-16T11:45:05.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.993 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:45:05.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:45:05.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347568+0000", 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347570+0000", 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347588+0000", 2024-09-16T11:45:05.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7914000000000002e-05 2024-09-16T11:45:05.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347592+0000", 2024-09-16T11:45:05.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3150000000000002e-06 2024-09-16T11:45:05.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:05.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375199+0000", 2024-09-16T11:45:05.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027606065999999999 2024-09-16T11:45:05.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:05.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375276+0000", 2024-09-16T11:45:05.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7527000000000005e-05 2024-09-16T11:45:05.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:05.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375359+0000", 2024-09-16T11:45:05.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3238000000000005e-05 2024-09-16T11:45:05.996 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.996 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:05.996 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:05.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.996 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16854 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.641381826, 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027945148999999999, 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16854 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347598+0000", 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:05.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347600+0000", 2024-09-16T11:45:05.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:45:05.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:05.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347606+0000", 2024-09-16T11:45:05.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0580000000000003e-06 2024-09-16T11:45:05.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:05.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:05.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:05.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347609+0000", 2024-09-16T11:45:05.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:45:05.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375386+0000", 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027776809 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375461+0000", 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4868000000000002e-05 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375542+0000", 2024-09-16T11:45:06.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1253000000000003e-05 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16855 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 4120576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.641365487000002, 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028111094999999999, 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.001 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16855 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347614+0000", 2024-09-16T11:45:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:45:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347616+0000", 2024-09-16T11:45:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:45:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347623+0000", 2024-09-16T11:45:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0099999999999998e-06 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347626+0000", 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2140000000000001e-06 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375569+0000", 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027942649999999999 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:06.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375643+0000", 2024-09-16T11:45:06.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3949999999999995e-05 2024-09-16T11:45:06.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375725+0000", 2024-09-16T11:45:06.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1992999999999999e-05 2024-09-16T11:45:06.005 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.005 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.005 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.005 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16856 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3801088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.005 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:45:06.006 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.641343415999998, 2024-09-16T11:45:06.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028268272000000001, 2024-09-16T11:45:06.006 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.006 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:06.006 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.006 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.006 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.006 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16856 2024-09-16T11:45:06.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.006 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:45:06.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:45:06.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347636+0000", 2024-09-16T11:45:06.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:45:06.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347638+0000", 2024-09-16T11:45:06.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.86e-06 2024-09-16T11:45:06.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347644+0000", 2024-09-16T11:45:06.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1739999999999997e-06 2024-09-16T11:45:06.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347656+0000", 2024-09-16T11:45:06.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2198000000000001e-05 2024-09-16T11:45:06.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375751+0000", 2024-09-16T11:45:06.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028094507000000001 2024-09-16T11:45:06.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:06.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375824+0000", 2024-09-16T11:45:06.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3178999999999998e-05 2024-09-16T11:45:06.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375904+0000", 2024-09-16T11:45:06.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9985000000000001e-05 2024-09-16T11:45:06.011 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.011 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.011 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.011 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16857 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.012 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:06.012 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.641316914000001, 2024-09-16T11:45:06.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028275403000000001, 2024-09-16T11:45:06.012 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.012 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:06.012 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.012 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.012 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.013 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16857 2024-09-16T11:45:06.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.013 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:06.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:06.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:06.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.46e-07 2024-09-16T11:45:06.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347666+0000", 2024-09-16T11:45:06.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8739999999999997e-06 2024-09-16T11:45:06.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347672+0000", 2024-09-16T11:45:06.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1059999999999996e-06 2024-09-16T11:45:06.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347676+0000", 2024-09-16T11:45:06.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:45:06.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375930+0000", 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028254042 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375937+0000", 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5519999999999999e-06 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.017 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16858 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.017 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:06.017 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.641299269000001, 2024-09-16T11:45:06.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028284592000000001, 2024-09-16T11:45:06.017 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.017 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:06.017 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.017 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.017 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.017 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16858 2024-09-16T11:45:06.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.017 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:45:06.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347683+0000", 2024-09-16T11:45:06.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6809999999999998e-06 2024-09-16T11:45:06.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347689+0000", 2024-09-16T11:45:06.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:45:06.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347695+0000", 2024-09-16T11:45:06.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0020000000000003e-06 2024-09-16T11:45:06.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375958+0000", 2024-09-16T11:45:06.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028263870999999999 2024-09-16T11:45:06.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375964+0000", 2024-09-16T11:45:06.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8209999999999998e-06 2024-09-16T11:45:06.020 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.020 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.021 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.021 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16992 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.021 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:45:06.021 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.639724580999996, 2024-09-16T11:45:06.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027704776, 2024-09-16T11:45:06.021 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.021 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:06.021 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.021 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.021 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.021 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16992 2024-09-16T11:45:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:45:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:45:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349255+0000", 2024-09-16T11:45:06.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:45:06.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349260+0000", 2024-09-16T11:45:06.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5589999999999996e-06 2024-09-16T11:45:06.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349267+0000", 2024-09-16T11:45:06.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4019999999999999e-06 2024-09-16T11:45:06.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349270+0000", 2024-09-16T11:45:06.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6780000000000002e-06 2024-09-16T11:45:06.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376953+0000", 2024-09-16T11:45:06.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682933 2024-09-16T11:45:06.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376959+0000", 2024-09-16T11:45:06.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:45:06.026 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.026 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.026 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.026 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16993 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.026 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:06.026 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.639703256000004, 2024-09-16T11:45:06.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027707908999999999, 2024-09-16T11:45:06.027 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.027 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:06.027 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.027 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.027 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.027 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16993 2024-09-16T11:45:06.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.029 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:06.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:06.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:06.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:45:06.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349278+0000", 2024-09-16T11:45:06.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.629e-06 2024-09-16T11:45:06.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349291+0000", 2024-09-16T11:45:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614e-05 2024-09-16T11:45:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349295+0000", 2024-09-16T11:45:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1189999999999999e-06 2024-09-16T11:45:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376978+0000", 2024-09-16T11:45:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682445 2024-09-16T11:45:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376984+0000", 2024-09-16T11:45:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7590000000000003e-06 2024-09-16T11:45:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16994 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:06.034 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.639679244999996, 2024-09-16T11:45:06.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027708753999999999, 2024-09-16T11:45:06.034 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.034 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:06.034 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.034 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.034 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.034 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16994 2024-09-16T11:45:06.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349302+0000", 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.722e-06 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349308+0000", 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:45:06.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349312+0000", 2024-09-16T11:45:06.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5310000000000002e-06 2024-09-16T11:45:06.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377003+0000", 2024-09-16T11:45:06.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027691308000000001 2024-09-16T11:45:06.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377008+0000", 2024-09-16T11:45:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4550000000000003e-06 2024-09-16T11:45:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16999 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.639536367000005, 2024-09-16T11:45:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027705270000000001, 2024-09-16T11:45:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16999 2024-09-16T11:45:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.46e-07 2024-09-16T11:45:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349445+0000", 2024-09-16T11:45:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.637e-06 2024-09-16T11:45:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349451+0000", 2024-09-16T11:45:06.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9959999999999999e-06 2024-09-16T11:45:06.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349454+0000", 2024-09-16T11:45:06.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.647e-06 2024-09-16T11:45:06.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377142+0000", 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027687942 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377148+0000", 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7019999999999998e-06 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17000 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.042 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:06.043 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.639520860000005, 2024-09-16T11:45:06.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.02771881, 2024-09-16T11:45:06.043 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.043 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:06.043 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.043 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.043 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.043 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17000 2024-09-16T11:45:06.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.043 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:06.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:06.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:06.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0600000000000001e-07 2024-09-16T11:45:06.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.045 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.045 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349460+0000", 2024-09-16T11:45:06.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.736e-06 2024-09-16T11:45:06.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.045 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.045 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349466+0000", 2024-09-16T11:45:06.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2419999999999997e-06 2024-09-16T11:45:06.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349470+0000", 2024-09-16T11:45:06.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5480000000000002e-06 2024-09-16T11:45:06.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377171+0000", 2024-09-16T11:45:06.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027701282000000001 2024-09-16T11:45:06.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377177+0000", 2024-09-16T11:45:06.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5960000000000003e-06 2024-09-16T11:45:06.047 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.047 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.047 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.047 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17001 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.047 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:06.047 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.639505110000002, 2024-09-16T11:45:06.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027739243, 2024-09-16T11:45:06.047 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17001 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.049 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:06.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.049 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:06.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:45:06.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.049 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349476+0000", 2024-09-16T11:45:06.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T11:45:06.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.050 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349482+0000", 2024-09-16T11:45:06.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2389999999999999e-06 2024-09-16T11:45:06.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.050 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.050 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349486+0000", 2024-09-16T11:45:06.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5980000000000001e-06 2024-09-16T11:45:06.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377206+0000", 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027720037999999999 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377213+0000", 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3440000000000003e-06 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.051 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17433 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 5898240~4888] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.052 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:45:06.052 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.603541456000002, 2024-09-16T11:45:06.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027709398, 2024-09-16T11:45:06.052 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.052 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:06.052 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.052 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.052 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.052 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17433 2024-09-16T11:45:06.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.052 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:45:06.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:45:06.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385438+0000", 2024-09-16T11:45:06.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:45:06.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385439+0000", 2024-09-16T11:45:06.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5120000000000001e-06 2024-09-16T11:45:06.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385445+0000", 2024-09-16T11:45:06.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4e-06 2024-09-16T11:45:06.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.055 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385448+0000", 2024-09-16T11:45:06.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1669999999999999e-06 2024-09-16T11:45:06.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.055 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412916+0000", 2024-09-16T11:45:06.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027468137 2024-09-16T11:45:06.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:06.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412988+0000", 2024-09-16T11:45:06.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1606000000000005e-05 2024-09-16T11:45:06.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413147+0000", 2024-09-16T11:45:06.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015924299999999999 2024-09-16T11:45:06.057 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.057 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.057 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.057 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17434 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6004736~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.057 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:06.057 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.603526865999996, 2024-09-16T11:45:06.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027777461, 2024-09-16T11:45:06.057 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.057 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:06.057 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.057 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17434 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:06.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:45:06.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385454+0000", 2024-09-16T11:45:06.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6729999999999999e-06 2024-09-16T11:45:06.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385459+0000", 2024-09-16T11:45:06.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4079999999999997e-06 2024-09-16T11:45:06.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385463+0000", 2024-09-16T11:45:06.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.269e-06 2024-09-16T11:45:06.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413157+0000", 2024-09-16T11:45:06.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027694611000000001 2024-09-16T11:45:06.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:06.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413193+0000", 2024-09-16T11:45:06.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5673999999999998e-05 2024-09-16T11:45:06.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413230+0000", 2024-09-16T11:45:06.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6454e-05 2024-09-16T11:45:06.062 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.062 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.062 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.062 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17435 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6324224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.062 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:45:06.062 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.603512713000001, 2024-09-16T11:45:06.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929059999999999, 2024-09-16T11:45:06.062 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.062 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:06.062 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.063 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.063 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.063 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17435 2024-09-16T11:45:06.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.063 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:45:06.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:45:06.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.064 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385467+0000", 2024-09-16T11:45:06.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:45:06.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.064 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385468+0000", 2024-09-16T11:45:06.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6300000000000001e-06 2024-09-16T11:45:06.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385474+0000", 2024-09-16T11:45:06.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1299999999999998e-06 2024-09-16T11:45:06.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385478+0000", 2024-09-16T11:45:06.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2880000000000001e-06 2024-09-16T11:45:06.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413257+0000", 2024-09-16T11:45:06.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027779608000000001 2024-09-16T11:45:06.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.067 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:06.067 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413331+0000", 2024-09-16T11:45:06.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3639000000000004e-05 2024-09-16T11:45:06.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.067 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.067 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413395+0000", 2024-09-16T11:45:06.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4424999999999994e-05 2024-09-16T11:45:06.067 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.067 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.067 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.068 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17436 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7176192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.068 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:06.068 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.603497051000005, 2024-09-16T11:45:06.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028016718999999999, 2024-09-16T11:45:06.068 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.068 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:06.068 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.068 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.068 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.068 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17436 2024-09-16T11:45:06.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.068 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:06.069 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:06.069 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:06.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:45:06.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385484+0000", 2024-09-16T11:45:06.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5179999999999999e-06 2024-09-16T11:45:06.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385489+0000", 2024-09-16T11:45:06.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7049999999999996e-06 2024-09-16T11:45:06.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385493+0000", 2024-09-16T11:45:06.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T11:45:06.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413422+0000", 2024-09-16T11:45:06.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929611999999999 2024-09-16T11:45:06.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413462+0000", 2024-09-16T11:45:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9585000000000001e-05 2024-09-16T11:45:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413499+0000", 2024-09-16T11:45:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6644999999999999e-05 2024-09-16T11:45:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17437 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6750208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:45:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.603482635999995, 2024-09-16T11:45:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028167168999999999, 2024-09-16T11:45:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17437 2024-09-16T11:45:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:45:06.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:45:06.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385497+0000", 2024-09-16T11:45:06.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:45:06.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385498+0000", 2024-09-16T11:45:06.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.592e-06 2024-09-16T11:45:06.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385504+0000", 2024-09-16T11:45:06.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4090000000000002e-06 2024-09-16T11:45:06.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385513+0000", 2024-09-16T11:45:06.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4460000000000007e-06 2024-09-16T11:45:06.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413526+0000", 2024-09-16T11:45:06.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028012866000000001 2024-09-16T11:45:06.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413601+0000", 2024-09-16T11:45:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5388000000000004e-05 2024-09-16T11:45:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413663+0000", 2024-09-16T11:45:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.211e-05 2024-09-16T11:45:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.078 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17438 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7495680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.078 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:06.078 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.603459877000006, 2024-09-16T11:45:06.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028224439, 2024-09-16T11:45:06.078 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.078 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:06.078 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.078 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.078 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.078 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17438 2024-09-16T11:45:06.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.078 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:06.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:06.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:06.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4900000000000001e-07 2024-09-16T11:45:06.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.080 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385521+0000", 2024-09-16T11:45:06.080 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:45:06.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.080 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385526+0000", 2024-09-16T11:45:06.080 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2680000000000002e-06 2024-09-16T11:45:06.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385529+0000", 2024-09-16T11:45:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2040000000000002e-06 2024-09-16T11:45:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413674+0000", 2024-09-16T11:45:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028144226000000001 2024-09-16T11:45:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413710+0000", 2024-09-16T11:45:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6665000000000002e-05 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413743+0000", 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3155999999999999e-05 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17439 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7921664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.603398722999998, 2024-09-16T11:45:06.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028326131000000001, 2024-09-16T11:45:06.083 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.083 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:06.083 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.083 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.083 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.083 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17439 2024-09-16T11:45:06.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.083 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.083 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.083 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.083 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:45:06.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.084 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:45:06.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.084 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385581+0000", 2024-09-16T11:45:06.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9299999999999998e-07 2024-09-16T11:45:06.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.084 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385583+0000", 2024-09-16T11:45:06.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6220000000000001e-06 2024-09-16T11:45:06.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385592+0000", 2024-09-16T11:45:06.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5220000000000007e-06 2024-09-16T11:45:06.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385595+0000", 2024-09-16T11:45:06.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8669999999999998e-06 2024-09-16T11:45:06.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413771+0000", 2024-09-16T11:45:06.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028175875999999999 2024-09-16T11:45:06.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:06.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413845+0000", 2024-09-16T11:45:06.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4382999999999996e-05 2024-09-16T11:45:06.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413906+0000", 2024-09-16T11:45:06.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1367999999999999e-05 2024-09-16T11:45:06.087 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.087 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.087 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17440 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.603381514999995, 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028391117, 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17440 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:06.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:45:06.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:06.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:45:06.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:06.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:06.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385598+0000", 2024-09-16T11:45:06.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385603+0000", 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385607+0000", 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4869999999999998e-06 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385609+0000", 2024-09-16T11:45:06.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2759999999999999e-06 2024-09-16T11:45:06.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:06.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413916+0000", 2024-09-16T11:45:06.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028307253000000001 2024-09-16T11:45:06.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:06.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413955+0000", 2024-09-16T11:45:06.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8652000000000002e-05 2024-09-16T11:45:06.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:06.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:06.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:06.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413989+0000", 2024-09-16T11:45:06.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3587999999999998e-05 2024-09-16T11:45:06.092 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.092 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.092 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.092 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:06.092 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:06.092 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:06.094 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:45:06.094 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:06.092+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:06.194 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:06.192+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:06.203 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:06.204 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:06.204 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:06.204 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:06.213 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:45:06.295 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:06.293+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:06.310 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:06.311 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:06.311 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:06.311 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:06.311 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:06.319 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:45:06.396 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:06.394+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:06.452 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T11:45:06.462 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098601368000004, 2024-09-16T11:45:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:45:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:45:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:45:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:45:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:45:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:45:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:45:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098579338999997, 2024-09-16T11:45:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:45:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:45:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098564573999994, 2024-09-16T11:45:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:45:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:45:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:45:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:45:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:45:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:45:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:45:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:45:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:45:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098544505999996, 2024-09-16T11:45:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:45:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:45:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:45:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:45:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:45:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:45:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:45:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:45:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:45:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:45:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098513670000003, 2024-09-16T11:45:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:45:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:45:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:45:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:45:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:45:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:45:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098494149999993, 2024-09-16T11:45:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:45:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:45:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:45:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:45:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:45:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:45:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:45:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:45:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098466571000003, 2024-09-16T11:45:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:45:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:45:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:45:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:45:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:45:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:45:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:45:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098447875999994, 2024-09-16T11:45:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:45:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:45:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:45:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:45:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:45:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:45:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:45:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:45:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:45:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:45:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098432867, 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:45:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:45:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:45:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:45:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:45:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:45:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:45:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:45:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:45:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:45:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:45:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098400768999994, 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:45:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:45:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:45:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:45:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:45:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:45:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:45:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:45:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:45:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:45:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:45:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:45:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:45:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:45:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098377150000005, 2024-09-16T11:45:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:45:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:45:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:45:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:45:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098362421000004, 2024-09-16T11:45:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:45:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:45:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:45:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:45:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:45:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:45:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:45:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:45:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:45:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:45:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098340597000004, 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:45:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:45:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:45:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:45:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:45:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:45:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:45:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:45:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098325947000006, 2024-09-16T11:45:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:45:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:45:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:45:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:45:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:45:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:45:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:45:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:45:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:45:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:45:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098304373999994, 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:45:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:45:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:45:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:45:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:45:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:45:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:45:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:45:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:45:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098284927999998, 2024-09-16T11:45:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:45:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:45:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:45:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:45:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:45:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:45:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:45:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:45:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098248230999999, 2024-09-16T11:45:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:45:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:45:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:45:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:45:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:45:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:45:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:45:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:45:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:45:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:45:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:45:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:45:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:45:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:45:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:45:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:45:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098226284999996, 2024-09-16T11:45:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:45:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:45:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:45:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:45:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:45:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:45:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:45:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:45:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:45:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098204398999997, 2024-09-16T11:45:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:45:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:45:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:45:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:45:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:45:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:45:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.098189680000004, 2024-09-16T11:45:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:45:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:45:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:45:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:45:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:45:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:45:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:45:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:45:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:45:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:06.554 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:06.554 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:45:06.557 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:45:06.557 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:06.553+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:06.656 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:06.654+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:06.695 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:06.696 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:06.696 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:06.696 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:06.704 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:45:06.709 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:45:06.710 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:45:06.725 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:45:06.725 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:45:06.739 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:45:06.740 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:45:06.740 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T11:45:06.756 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:45:06.757 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:45:06.757 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:06.755+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:06.771 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:45:06.771 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:45:06.786 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:45:06.786 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:45:06.801 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:45:06.801 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:45:06.804 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:45:06.804 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:45:06.844 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:06.844 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:06.851 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:45:06.857 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:06.856+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:06.948 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:06.948 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:06.956 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:45:06.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-16T11:45:06.958 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:06.955+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:07.053 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:07.054 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:07.059 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:07.056+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:07.062 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:45:07.160 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:07.158+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:07.200 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:07.200 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:07.209 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:45:07.293 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:07.291+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:07.347 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:07.347 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:07.355 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:45:07.397 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:07.394+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:07.462 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:07.461+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:07.495 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:07.495 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:07.503 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:45:07.563 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:07.561+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:07.608 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:07.608 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:07.617 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:45:07.664 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:07.662+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:07.717 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:07.717 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:07.725 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:45:07.769 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:07.768+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:07.824 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:07.825 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:07.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:45:07.867 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:07.865+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:07.968 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:07.967+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:07.970 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:07.970 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:07.979 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:45:08.068 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:08.066+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:08.117 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:08.117 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:08.126 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:45:08.170 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:08.169+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:08.224 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:08.224 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:08.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:45:08.270 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:08.268+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:08.369 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:08.369 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:08.370 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:08.369+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:08.377 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:45:08.471 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:08.470+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:08.515 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:08.516 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:08.524 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:45:08.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:45:08.608 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:08.607+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:08.669 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:08.669 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:08.671 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:08.669+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:08.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:45:08.772 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:08.771+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:08.806 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:45:08.819 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:08.819 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:08.829 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:45:08.914 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:08.913+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:08.921 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:45:08.921 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:45:08.937 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:45:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:45:08.952 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:45:08.952 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:45:08.953 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T11:45:08.968 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:45:08.968 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:45:08.973 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:45:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:45:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:45:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16853 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3375104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:45:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.623154197999995, 2024-09-16T11:45:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027791907000000001, 2024-09-16T11:45:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16853 2024-09-16T11:45:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:45:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:45:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347568+0000", 2024-09-16T11:45:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:45:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347570+0000", 2024-09-16T11:45:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:45:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347588+0000", 2024-09-16T11:45:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7914000000000002e-05 2024-09-16T11:45:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347592+0000", 2024-09-16T11:45:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3150000000000002e-06 2024-09-16T11:45:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375199+0000", 2024-09-16T11:45:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027606065999999999 2024-09-16T11:45:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375276+0000", 2024-09-16T11:45:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7527000000000005e-05 2024-09-16T11:45:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375359+0000", 2024-09-16T11:45:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3238000000000005e-05 2024-09-16T11:45:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16854 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:45:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.623124501000007, 2024-09-16T11:45:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027945148999999999, 2024-09-16T11:45:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16854 2024-09-16T11:45:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:45:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:45:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347598+0000", 2024-09-16T11:45:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:45:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347600+0000", 2024-09-16T11:45:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:45:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:08.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347606+0000", 2024-09-16T11:45:08.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0580000000000003e-06 2024-09-16T11:45:08.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:08.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347609+0000", 2024-09-16T11:45:08.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:45:08.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:08.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375386+0000", 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027776809 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375461+0000", 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4868000000000002e-05 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375542+0000", 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1253000000000003e-05 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:08.982 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16855 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 4120576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.623108161999994, 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028111094999999999, 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16855 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.983 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347614+0000", 2024-09-16T11:45:08.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:45:08.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:08.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347616+0000", 2024-09-16T11:45:08.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:45:08.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:08.985 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347623+0000", 2024-09-16T11:45:08.985 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0099999999999998e-06 2024-09-16T11:45:08.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.985 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347626+0000", 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2140000000000001e-06 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375569+0000", 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027942649999999999 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375643+0000", 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3949999999999995e-05 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375725+0000", 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1992999999999999e-05 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16856 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3801088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.623086091000005, 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028268272000000001, 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:08.987 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:08.988 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:08.988 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:08.988 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:08.988 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16856 2024-09-16T11:45:08.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.988 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:08.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:08.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:45:08.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:08.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347636+0000", 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347638+0000", 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.86e-06 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:08.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347644+0000", 2024-09-16T11:45:08.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1739999999999997e-06 2024-09-16T11:45:08.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:08.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347656+0000", 2024-09-16T11:45:08.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2198000000000001e-05 2024-09-16T11:45:08.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:08.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375751+0000", 2024-09-16T11:45:08.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028094507000000001 2024-09-16T11:45:08.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:08.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375824+0000", 2024-09-16T11:45:08.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3178999999999998e-05 2024-09-16T11:45:08.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:08.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375904+0000", 2024-09-16T11:45:08.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9985000000000001e-05 2024-09-16T11:45:08.991 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:08.992 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:08.992 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:08.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.992 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16857 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:08.992 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:08.992 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.623059588999993, 2024-09-16T11:45:08.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028275403000000001, 2024-09-16T11:45:08.992 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:08.992 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:08.992 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:08.992 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:08.992 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:08.993 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16857 2024-09-16T11:45:08.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.993 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:08.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:08.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:08.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:08.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.993 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:08.993 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:08.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:08.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.46e-07 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347666+0000", 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8739999999999997e-06 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347672+0000", 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1059999999999996e-06 2024-09-16T11:45:08.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347676+0000", 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375930+0000", 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028254042 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375937+0000", 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5519999999999999e-06 2024-09-16T11:45:08.995 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16858 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.623041943999993, 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028284592000000001, 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16858 2024-09-16T11:45:08.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.997 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:08.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:08.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:08.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:08.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:08.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:08.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:08.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:08.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:08.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:45:08.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:08.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347683+0000", 2024-09-16T11:45:08.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6809999999999998e-06 2024-09-16T11:45:08.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:08.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347689+0000", 2024-09-16T11:45:08.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:45:08.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:08.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347695+0000", 2024-09-16T11:45:08.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0020000000000003e-06 2024-09-16T11:45:08.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:08.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:08.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:08.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375958+0000", 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028263870999999999 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375964+0000", 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8209999999999998e-06 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16992 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:45:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.621467256000003, 2024-09-16T11:45:09.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027704776, 2024-09-16T11:45:09.001 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.001 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:09.001 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.001 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.001 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.001 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16992 2024-09-16T11:45:09.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.001 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.001 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.001 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:45:09.001 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349255+0000", 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349260+0000", 2024-09-16T11:45:09.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5589999999999996e-06 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349267+0000", 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4019999999999999e-06 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349270+0000", 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6780000000000002e-06 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376953+0000", 2024-09-16T11:45:09.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682933 2024-09-16T11:45:09.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376959+0000", 2024-09-16T11:45:09.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:45:09.004 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.004 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.004 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.004 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16993 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.004 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.621445930999997, 2024-09-16T11:45:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027707908999999999, 2024-09-16T11:45:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16993 2024-09-16T11:45:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349278+0000", 2024-09-16T11:45:09.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.629e-06 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349291+0000", 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614e-05 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349295+0000", 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1189999999999999e-06 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376978+0000", 2024-09-16T11:45:09.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682445 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376984+0000", 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7590000000000003e-06 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16994 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.621421920000003, 2024-09-16T11:45:09.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027708753999999999, 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16994 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:09.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:09.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:45:09.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.010 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.010 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349302+0000", 2024-09-16T11:45:09.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.722e-06 2024-09-16T11:45:09.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349308+0000", 2024-09-16T11:45:09.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:45:09.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349312+0000", 2024-09-16T11:45:09.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5310000000000002e-06 2024-09-16T11:45:09.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377003+0000", 2024-09-16T11:45:09.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027691308000000001 2024-09-16T11:45:09.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.012 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377008+0000", 2024-09-16T11:45:09.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4550000000000003e-06 2024-09-16T11:45:09.012 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.012 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.012 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.012 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16999 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.013 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:09.013 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.621279041999998, 2024-09-16T11:45:09.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027705270000000001, 2024-09-16T11:45:09.013 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.013 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:09.013 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.013 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.013 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.013 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16999 2024-09-16T11:45:09.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.013 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.46e-07 2024-09-16T11:45:09.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349445+0000", 2024-09-16T11:45:09.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.637e-06 2024-09-16T11:45:09.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349451+0000", 2024-09-16T11:45:09.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9959999999999999e-06 2024-09-16T11:45:09.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349454+0000", 2024-09-16T11:45:09.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.647e-06 2024-09-16T11:45:09.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377142+0000", 2024-09-16T11:45:09.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027687942 2024-09-16T11:45:09.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377148+0000", 2024-09-16T11:45:09.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7019999999999998e-06 2024-09-16T11:45:09.016 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.016 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.017 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.017 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17000 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.017 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:09.017 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.621263534999997, 2024-09-16T11:45:09.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.02771881, 2024-09-16T11:45:09.017 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.017 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:09.017 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.017 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.017 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.017 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17000 2024-09-16T11:45:09.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.018 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.018 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.018 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:09.018 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.018 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.018 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:09.018 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:09.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0600000000000001e-07 2024-09-16T11:45:09.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349460+0000", 2024-09-16T11:45:09.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.736e-06 2024-09-16T11:45:09.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349466+0000", 2024-09-16T11:45:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2419999999999997e-06 2024-09-16T11:45:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349470+0000", 2024-09-16T11:45:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5480000000000002e-06 2024-09-16T11:45:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377171+0000", 2024-09-16T11:45:09.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027701282000000001 2024-09-16T11:45:09.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.021 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.021 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377177+0000", 2024-09-16T11:45:09.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5960000000000003e-06 2024-09-16T11:45:09.021 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.021 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.021 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.021 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17001 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.021 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:09.022 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.621247784999994, 2024-09-16T11:45:09.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027739243, 2024-09-16T11:45:09.022 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.022 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:09.022 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.022 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.022 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.022 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17001 2024-09-16T11:45:09.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.022 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349476+0000", 2024-09-16T11:45:09.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T11:45:09.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349482+0000", 2024-09-16T11:45:09.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2389999999999999e-06 2024-09-16T11:45:09.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349486+0000", 2024-09-16T11:45:09.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5980000000000001e-06 2024-09-16T11:45:09.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377206+0000", 2024-09-16T11:45:09.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027720037999999999 2024-09-16T11:45:09.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377213+0000", 2024-09-16T11:45:09.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3440000000000003e-06 2024-09-16T11:45:09.025 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.025 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.025 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17433 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 5898240~4888] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.585284130999995, 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027709398, 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17433 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385438+0000", 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:45:09.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385439+0000", 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5120000000000001e-06 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385445+0000", 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4e-06 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385448+0000", 2024-09-16T11:45:09.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1669999999999999e-06 2024-09-16T11:45:09.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412916+0000", 2024-09-16T11:45:09.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027468137 2024-09-16T11:45:09.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:09.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412988+0000", 2024-09-16T11:45:09.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1606000000000005e-05 2024-09-16T11:45:09.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413147+0000", 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015924299999999999 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17434 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6004736~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.585269541000002, 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027777461, 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:09.030 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17434 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:09.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:09.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:45:09.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385454+0000", 2024-09-16T11:45:09.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6729999999999999e-06 2024-09-16T11:45:09.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385459+0000", 2024-09-16T11:45:09.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4079999999999997e-06 2024-09-16T11:45:09.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.033 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385463+0000", 2024-09-16T11:45:09.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.269e-06 2024-09-16T11:45:09.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413157+0000", 2024-09-16T11:45:09.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027694611000000001 2024-09-16T11:45:09.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:09.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413193+0000", 2024-09-16T11:45:09.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5673999999999998e-05 2024-09-16T11:45:09.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413230+0000", 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6454e-05 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17435 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6324224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.585255387999993, 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929059999999999, 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.035 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17435 2024-09-16T11:45:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:45:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:45:09.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385467+0000", 2024-09-16T11:45:09.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:45:09.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385468+0000", 2024-09-16T11:45:09.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6300000000000001e-06 2024-09-16T11:45:09.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385474+0000", 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1299999999999998e-06 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385478+0000", 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2880000000000001e-06 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413257+0000", 2024-09-16T11:45:09.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027779608000000001 2024-09-16T11:45:09.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:09.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413331+0000", 2024-09-16T11:45:09.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3639000000000004e-05 2024-09-16T11:45:09.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413395+0000", 2024-09-16T11:45:09.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4424999999999994e-05 2024-09-16T11:45:09.040 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.040 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.040 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.040 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17436 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7176192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.040 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:09.040 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.585239725999998, 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028016718999999999, 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17436 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:09.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:09.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:45:09.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385484+0000", 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5179999999999999e-06 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385489+0000", 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7049999999999996e-06 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385493+0000", 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413422+0000", 2024-09-16T11:45:09.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929611999999999 2024-09-16T11:45:09.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:09.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413462+0000", 2024-09-16T11:45:09.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9585000000000001e-05 2024-09-16T11:45:09.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.044 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.044 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413499+0000", 2024-09-16T11:45:09.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6644999999999999e-05 2024-09-16T11:45:09.044 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.045 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.045 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.045 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17437 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6750208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.045 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:45:09.045 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.585225311000002, 2024-09-16T11:45:09.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028167168999999999, 2024-09-16T11:45:09.045 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.045 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:09.045 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.045 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.045 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.046 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17437 2024-09-16T11:45:09.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.046 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:45:09.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:45:09.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385497+0000", 2024-09-16T11:45:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:45:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385498+0000", 2024-09-16T11:45:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.592e-06 2024-09-16T11:45:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385504+0000", 2024-09-16T11:45:09.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4090000000000002e-06 2024-09-16T11:45:09.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385513+0000", 2024-09-16T11:45:09.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4460000000000007e-06 2024-09-16T11:45:09.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413526+0000", 2024-09-16T11:45:09.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028012866000000001 2024-09-16T11:45:09.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:09.049 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413601+0000", 2024-09-16T11:45:09.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5388000000000004e-05 2024-09-16T11:45:09.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.049 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413663+0000", 2024-09-16T11:45:09.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.211e-05 2024-09-16T11:45:09.049 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.049 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.049 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17438 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7495680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.585202551999998, 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028224439, 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17438 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.050 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:09.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:09.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:09.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4900000000000001e-07 2024-09-16T11:45:09.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385521+0000", 2024-09-16T11:45:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:45:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385526+0000", 2024-09-16T11:45:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2680000000000002e-06 2024-09-16T11:45:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385529+0000", 2024-09-16T11:45:09.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2040000000000002e-06 2024-09-16T11:45:09.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413674+0000", 2024-09-16T11:45:09.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028144226000000001 2024-09-16T11:45:09.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:09.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413710+0000", 2024-09-16T11:45:09.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6665000000000002e-05 2024-09-16T11:45:09.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413743+0000", 2024-09-16T11:45:09.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3155999999999999e-05 2024-09-16T11:45:09.054 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.054 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.054 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.054 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17439 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7921664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.054 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:45:09.054 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.585141398000005, 2024-09-16T11:45:09.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028326131000000001, 2024-09-16T11:45:09.054 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17439 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.055 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385581+0000", 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9299999999999998e-07 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385583+0000", 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6220000000000001e-06 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385592+0000", 2024-09-16T11:45:09.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5220000000000007e-06 2024-09-16T11:45:09.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385595+0000", 2024-09-16T11:45:09.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8669999999999998e-06 2024-09-16T11:45:09.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413771+0000", 2024-09-16T11:45:09.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028175875999999999 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413845+0000", 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4382999999999996e-05 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413906+0000", 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1367999999999999e-05 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17440 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 73.585124190000002, 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028391117, 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17440 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:09.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385598+0000", 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:45:09.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385603+0000", 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385607+0000", 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4869999999999998e-06 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385609+0000", 2024-09-16T11:45:09.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2759999999999999e-06 2024-09-16T11:45:09.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:09.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413916+0000", 2024-09-16T11:45:09.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028307253000000001 2024-09-16T11:45:09.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:09.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413955+0000", 2024-09-16T11:45:09.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8652000000000002e-05 2024-09-16T11:45:09.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:09.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:09.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:09.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413989+0000", 2024-09-16T11:45:09.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3587999999999998e-05 2024-09-16T11:45:09.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.063 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:09.063 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:09.063 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:09.065 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:45:09.066 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:45:09.066 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:45:09.066 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:45:09.067 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:45:09.069 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:45:09.069 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:45:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:45:09.070 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:45:09.071 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:09.062+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:09.165 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:09.164+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:09.172 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:09.172 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:09.180 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:45:09.266 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:09.264+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:09.278 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:09.278 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:09.278 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:09.278 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:09.278 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:09.286 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:45:09.370 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:09.368+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:09.426 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:09.426 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062543149000007, 2024-09-16T11:45:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:45:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:45:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:45:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:45:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:45:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:45:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:45:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:45:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:45:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:45:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.06252112, 2024-09-16T11:45:09.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:45:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:45:09.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:45:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:45:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:45:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:45:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:45:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:45:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:45:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:45:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:45:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:45:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:45:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062506354999996, 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.436 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:45:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:45:09.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:45:09.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:45:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:45:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:45:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:09.440 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062486286999999, 2024-09-16T11:45:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:45:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:45:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:09.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:45:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:45:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:45:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:45:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:45:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:45:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:45:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:45:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:45:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:45:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:45:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:45:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062455451000005, 2024-09-16T11:45:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:45:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:09.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:45:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:45:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:45:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:45:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:45:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:45:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:45:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062435930999996, 2024-09-16T11:45:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:45:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:45:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:45:09.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:45:09.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:45:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:45:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:45:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:45:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:45:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:45:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:45:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062408352000006, 2024-09-16T11:45:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:45:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.454 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:09.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:45:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:45:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:45:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:45:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:45:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:45:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:45:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:45:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:45:09.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062389656999997, 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.462 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062374648000002, 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:45:09.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:45:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:45:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:45:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:45:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:45:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:45:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:45:09.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062342549999997, 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:45:09.467 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:45:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:45:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:45:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:45:09.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:45:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:45:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:45:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:45:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062318930999993, 2024-09-16T11:45:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:45:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:45:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.472 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062304202000007, 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:45:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:45:09.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:45:09.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:45:09.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:45:09.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.480 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062282378000006, 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:45:09.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:45:09.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062267727999995, 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.485 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:45:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:45:09.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:45:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:45:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:45:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:45:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:45:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:45:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:45:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:45:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.489 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062246154999997, 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:45:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:45:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:45:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:45:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:45:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:45:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:45:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:45:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:45:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:45:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:45:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:45:09.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062226709000001, 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:45:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:45:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:45:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:45:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:45:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:45:09.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:45:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:45:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:09.498 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062190012000002, 2024-09-16T11:45:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:45:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:45:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:45:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062168065999998, 2024-09-16T11:45:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:45:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.504 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:09.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:45:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:45:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:45:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:45:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.508 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062146179999999, 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:45:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:45:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:45:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:45:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:45:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:45:09.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:45:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:45:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:45:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:45:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:45:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:45:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.062131461000007, 2024-09-16T11:45:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:45:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.513 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:09.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:45:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:45:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:45:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:45:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:45:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:45:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.517 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:09.519 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:45:09.519 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:09.517+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:09.544 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T11:45:09.620 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:09.618+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:09.664 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:09.664 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:09.664 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:09.664 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:09.673 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:45:09.721 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:09.720+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:09.810 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:09.811 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:09.819 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:45:09.822 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:09.821+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:09.919 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:09.919 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:09.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:09.919 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19943 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2416640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390455334999999, 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027947257999999999, 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19943 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:45:09.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:45:09.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:45:09.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5999999999999999e-07 2024-09-16T11:45:09.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526297+0000", 2024-09-16T11:45:09.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8059999999999999e-06 2024-09-16T11:45:09.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526303+0000", 2024-09-16T11:45:09.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6579999999999999e-06 2024-09-16T11:45:09.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526306+0000", 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3969999999999998e-06 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554165+0000", 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027858519000000002 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554201+0000", 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5908000000000002e-05 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554242+0000", 2024-09-16T11:45:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1510000000000001e-05 2024-09-16T11:45:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19944 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2842624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:45:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390440440000006, 2024-09-16T11:45:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028052639000000001, 2024-09-16T11:45:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19944 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526312+0000", 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526317+0000", 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5910000000000003e-06 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526321+0000", 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2339999999999999e-06 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554263+0000", 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027942181 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554307+0000", 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4499999999999997e-05 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554362+0000", 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5078999999999999e-05 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19945 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3162112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:45:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390425884999999, 2024-09-16T11:45:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028145673999999999, 2024-09-16T11:45:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19945 2024-09-16T11:45:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526325+0000", 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526326+0000", 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.733e-06 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526332+0000", 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526336+0000", 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8410000000000002e-06 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554389+0000", 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028053576 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554427+0000", 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7299999999999999e-05 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554470+0000", 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3195000000000002e-05 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19946 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3268608~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:45:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390410306000007, 2024-09-16T11:45:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028232587, 2024-09-16T11:45:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19946 2024-09-16T11:45:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:45:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:45:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526346+0000", 2024-09-16T11:45:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6459999999999998e-06 2024-09-16T11:45:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526352+0000", 2024-09-16T11:45:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1070000000000001e-06 2024-09-16T11:45:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526355+0000", 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554493+0000", 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.02813746 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554530+0000", 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6881999999999997e-05 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554572+0000", 2024-09-16T11:45:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2688999999999997e-05 2024-09-16T11:45:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19947 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4014080~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:45:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390390952000004, 2024-09-16T11:45:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028315646999999999, 2024-09-16T11:45:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19947 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526360+0000", 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.01e-07 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526361+0000", 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8300000000000001e-06 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526369+0000", 2024-09-16T11:45:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:45:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526377+0000", 2024-09-16T11:45:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0909999999999993e-06 2024-09-16T11:45:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554593+0000", 2024-09-16T11:45:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028216657999999999 2024-09-16T11:45:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554632+0000", 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8828000000000002e-05 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554675+0000", 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2710000000000003e-05 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19948 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3588096~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390368553000002, 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028394545, 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19948 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:45:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9200000000000001e-07 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526384+0000", 2024-09-16T11:45:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526391+0000", 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5150000000000003e-06 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526394+0000", 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4240000000000002e-06 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554695+0000", 2024-09-16T11:45:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028301218999999999 2024-09-16T11:45:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554731+0000", 2024-09-16T11:45:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6298999999999999e-05 2024-09-16T11:45:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554776+0000", 2024-09-16T11:45:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4675999999999997e-05 2024-09-16T11:45:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19949 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1351680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390352281000006, 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028490081, 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19949 2024-09-16T11:45:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526400+0000", 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7099999999999999e-06 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526406+0000", 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5140000000000001e-06 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526409+0000", 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5360000000000001e-06 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554801+0000", 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028392378999999999 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554843+0000", 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1056000000000001e-05 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554888+0000", 2024-09-16T11:45:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466999999999997e-05 2024-09-16T11:45:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19950 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1777664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:45:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390337285000001, 2024-09-16T11:45:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028661844999999998, 2024-09-16T11:45:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19950 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526415+0000", 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.776e-06 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526422+0000", 2024-09-16T11:45:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6930000000000001e-06 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526425+0000", 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554923+0000", 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028498102000000001 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555000+0000", 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6619999999999995e-05 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555075+0000", 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4931999999999993e-05 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19951 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2097152~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:45:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390321474000004, 2024-09-16T11:45:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028755718, 2024-09-16T11:45:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19951 2024-09-16T11:45:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:45:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526431+0000", 2024-09-16T11:45:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526441+0000", 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7790000000000007e-06 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526445+0000", 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.634e-06 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555100+0000", 2024-09-16T11:45:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654722000000001 2024-09-16T11:45:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555138+0000", 2024-09-16T11:45:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7996e-05 2024-09-16T11:45:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555184+0000", 2024-09-16T11:45:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6427999999999997e-05 2024-09-16T11:45:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19952 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2203648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390300913999994, 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028901157, 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19952 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526450+0000", 2024-09-16T11:45:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:45:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526452+0000", 2024-09-16T11:45:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9039999999999999e-06 2024-09-16T11:45:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526465+0000", 2024-09-16T11:45:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3559e-05 2024-09-16T11:45:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526470+0000", 2024-09-16T11:45:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-09-16T11:45:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555217+0000", 2024-09-16T11:45:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028747287999999999 2024-09-16T11:45:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555272+0000", 2024-09-16T11:45:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4885000000000001e-05 2024-09-16T11:45:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555350+0000", 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8428000000000002e-05 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19953 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2523136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390276322000005, 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029004309999999998, 2024-09-16T11:45:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19953 2024-09-16T11:45:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:45:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:45:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:45:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-16T11:45:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526476+0000", 2024-09-16T11:45:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9810000000000002e-06 2024-09-16T11:45:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526482+0000", 2024-09-16T11:45:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:45:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526485+0000", 2024-09-16T11:45:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2760000000000001e-06 2024-09-16T11:45:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555383+0000", 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028898113 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555433+0000", 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9753999999999998e-05 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555478+0000", 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5006999999999998e-05 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19954 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2949120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:45:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390261324999997, 2024-09-16T11:45:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029165054999999999, 2024-09-16T11:45:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19954 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:45:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1799999999999995e-07 2024-09-16T11:45:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526491+0000", 2024-09-16T11:45:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7320000000000001e-06 2024-09-16T11:45:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526497+0000", 2024-09-16T11:45:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6869999999999999e-06 2024-09-16T11:45:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526500+0000", 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555513+0000", 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029012626 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555576+0000", 2024-09-16T11:45:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3288999999999997e-05 2024-09-16T11:45:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555654+0000", 2024-09-16T11:45:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7861999999999994e-05 2024-09-16T11:45:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19955 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3375104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:45:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390242022999999, 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029241979000000001, 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19955 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:45:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526509+0000", 2024-09-16T11:45:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-09-16T11:45:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526510+0000", 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.844e-06 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526516+0000", 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9039999999999997e-06 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526520+0000", 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3639999999999999e-06 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555669+0000", 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029149628 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555706+0000", 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7017000000000002e-05 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555750+0000", 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3856999999999997e-05 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19956 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3694592~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390227068000002, 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029398799999999999, 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19956 2024-09-16T11:45:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:45:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:45:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526524+0000", 2024-09-16T11:45:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:45:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526525+0000", 2024-09-16T11:45:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-16T11:45:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526531+0000", 2024-09-16T11:45:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6010000000000002e-06 2024-09-16T11:45:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526535+0000", 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.597e-06 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555785+0000", 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029250235999999999 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555845+0000", 2024-09-16T11:45:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0393000000000003e-05 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555922+0000", 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6710000000000002e-05 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19957 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4120576~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.390211914999995, 2024-09-16T11:45:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029481157000000001, 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19957 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526539+0000", 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526540+0000", 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.759e-06 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526546+0000", 2024-09-16T11:45:09.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1079999999999998e-06 2024-09-16T11:45:09.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526550+0000", 2024-09-16T11:45:09.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.269e-06 2024-09-16T11:45:09.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555939+0000", 2024-09-16T11:45:09.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029389444000000001 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555977+0000", 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7487999999999998e-05 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.556019+0000", 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2756999999999997e-05 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:28959 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.907355503000005, 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031145285000000002, 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 28959 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009396+0000", 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.296e-06 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030864+0000", 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.021468451 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030892+0000", 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7555999999999998e-05 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030902+0000", 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9550000000000004e-06 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030979+0000", 2024-09-16T11:45:09.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7137000000000004e-05 2024-09-16T11:45:09.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031051+0000", 2024-09-16T11:45:09.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1475000000000003e-05 2024-09-16T11:45:09.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:45:09.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031389+0000", 2024-09-16T11:45:09.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033827100000000001 2024-09-16T11:45:09.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.039264+0000", 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078751700000000008 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040484+0000", 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012199579999999999 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040517+0000", 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2960999999999998e-05 2024-09-16T11:45:09.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040540+0000", 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3054999999999999e-05 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30605 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 3915776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.673049243999998, 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938655999999999, 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.991 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30605 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243704+0000", 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2199999999999999e-06 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243710+0000", 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6340000000000003e-06 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243714+0000", 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5030000000000002e-06 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271450+0000", 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027736035999999999 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271536+0000", 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6521999999999996e-05 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271640+0000", 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103341 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30607 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4046848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:45:09.995 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:45:09.996 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.673026089000004, 2024-09-16T11:45:09.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028130446, 2024-09-16T11:45:09.996 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:09.996 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:09.996 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:09.996 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:09.996 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:09.996 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30607 2024-09-16T11:45:09.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.996 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:09.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:45:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:45:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243725+0000", 2024-09-16T11:45:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2399999999999998e-07 2024-09-16T11:45:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:09.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243727+0000", 2024-09-16T11:45:09.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3379999999999999e-06 2024-09-16T11:45:09.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:09.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243733+0000", 2024-09-16T11:45:09.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4280000000000001e-06 2024-09-16T11:45:09.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:09.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243737+0000", 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8990000000000003e-06 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271676+0000", 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938481000000001 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271757+0000", 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1396999999999993e-05 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:09.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271855+0000", 2024-09-16T11:45:10.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7379000000000004e-05 2024-09-16T11:45:10.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.000 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.000 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30608 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:45:10.000 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:45:10.000 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.673009238000006, 2024-09-16T11:45:10.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028323879, 2024-09-16T11:45:10.000 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.000 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30608 2024-09-16T11:45:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:45:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:45:10.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:45:10.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:45:10.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243743+0000", 2024-09-16T11:45:10.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.762e-06 2024-09-16T11:45:10.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243749+0000", 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5670000000000002e-06 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243758+0000", 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8309999999999996e-06 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271879+0000", 2024-09-16T11:45:10.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028120501999999999 2024-09-16T11:45:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271965+0000", 2024-09-16T11:45:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5755999999999997e-05 2024-09-16T11:45:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.272065+0000", 2024-09-16T11:45:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000100101 2024-09-16T11:45:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.004 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.005 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:56045 4.15 4:ab2ce1b7:::rbd_data.11c1b01b799a.0000000000000020:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-16T11:45:10.005 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:45:10.005 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.737725986999997, 2024-09-16T11:45:10.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032248165000000002, 2024-09-16T11:45:10.005 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.005 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.005 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.005 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.005 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.005 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 56045 2024-09-16T11:45:10.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.006 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:45:10.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:45:10.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179026+0000", 2024-09-16T11:45:10.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.556e-06 2024-09-16T11:45:10.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179032+0000", 2024-09-16T11:45:10.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5379999999999999e-06 2024-09-16T11:45:10.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179045+0000", 2024-09-16T11:45:10.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.277e-05 2024-09-16T11:45:10.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179067+0000", 2024-09-16T11:45:10.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.243e-05 2024-09-16T11:45:10.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179120+0000", 2024-09-16T11:45:10.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2726000000000003e-05 2024-09-16T11:45:10.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179279+0000", 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015865899999999999 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.185163+0000", 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0058837639999999997 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211197+0000", 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026034130999999999 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:10.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211206+0000", 2024-09-16T11:45:10.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2539999999999998e-06 2024-09-16T11:45:10.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211245+0000", 2024-09-16T11:45:10.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8924999999999997e-05 2024-09-16T11:45:10.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211272+0000", 2024-09-16T11:45:10.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7412000000000001e-05 2024-09-16T11:45:10.010 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.011 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.011 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.011 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.011 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.011 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:10.012 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:45:10.013 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:10.010+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:10.113 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:10.111+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:10.151 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:10.151 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:10.151 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:10.159 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:45:10.214 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:10.213+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:10.297 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:10.297 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:10.298 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:10.298 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:10.298 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:10.306 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:45:10.349 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:10.347+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:10.405 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:10.405 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:10.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:10.405 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:10.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.405 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19282 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:10.405 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:45:10.405 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.929463064000004, 2024-09-16T11:45:10.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521151000000002, 2024-09-16T11:45:10.405 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19282 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472838+0000", 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472848+0000", 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9799999999999993e-06 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472858+0000", 2024-09-16T11:45:10.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9429999999999995e-06 2024-09-16T11:45:10.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472862+0000", 2024-09-16T11:45:10.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4460000000000003e-06 2024-09-16T11:45:10.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501277+0000", 2024-09-16T11:45:10.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028414900999999999 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501322+0000", 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4880999999999999e-05 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501358+0000", 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6245000000000003e-05 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19286 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7979008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.929090131999999, 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521860999999999, 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19286 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473211+0000", 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473214+0000", 2024-09-16T11:45:10.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1080000000000001e-06 2024-09-16T11:45:10.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473226+0000", 2024-09-16T11:45:10.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2441e-05 2024-09-16T11:45:10.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473234+0000", 2024-09-16T11:45:10.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729999999999998e-06 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501592+0000", 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028358491999999999 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501671+0000", 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8846999999999999e-05 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501732+0000", 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1118000000000007e-05 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20619 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7610368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.841740963999996, 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028500152000000001, 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20619 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560560+0000", 2024-09-16T11:45:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9499999999999998e-07 2024-09-16T11:45:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560561+0000", 2024-09-16T11:45:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.071e-06 2024-09-16T11:45:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560565+0000", 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560567+0000", 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588949+0000", 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381878999999999 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588989+0000", 2024-09-16T11:45:10.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0649000000000003e-05 2024-09-16T11:45:10.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589059+0000", 2024-09-16T11:45:10.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0239000000000003e-05 2024-09-16T11:45:10.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.418 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.418 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20620 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7716864~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:10.419 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:45:10.419 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.841731564, 2024-09-16T11:45:10.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028627621999999998, 2024-09-16T11:45:10.419 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.419 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.419 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20620 2024-09-16T11:45:10.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.419 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:45:10.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:45:10.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562377+0000", 2024-09-16T11:45:10.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018081729999999999 2024-09-16T11:45:10.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562380+0000", 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0469999999999998e-06 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562386+0000", 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2040000000000002e-06 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562390+0000", 2024-09-16T11:45:10.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:45:10.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589092+0000", 2024-09-16T11:45:10.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026701798999999998 2024-09-16T11:45:10.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589139+0000", 2024-09-16T11:45:10.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7320000000000001e-05 2024-09-16T11:45:10.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589196+0000", 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7201999999999999e-05 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41954 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.988025774, 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028594441000000002, 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41954 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:45:10.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414275+0000", 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414277+0000", 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414282+0000", 2024-09-16T11:45:10.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7559999999999996e-06 2024-09-16T11:45:10.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414286+0000", 2024-09-16T11:45:10.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:45:10.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442778+0000", 2024-09-16T11:45:10.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028492237 2024-09-16T11:45:10.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442816+0000", 2024-09-16T11:45:10.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8198000000000003e-05 2024-09-16T11:45:10.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442869+0000", 2024-09-16T11:45:10.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2589000000000001e-05 2024-09-16T11:45:10.427 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.427 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.427 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41955 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.988010715999998, 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028687865, 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41955 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:10.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:10.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:10.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:45:10.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414292+0000", 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414299+0000", 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4279999999999999e-06 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414303+0000", 2024-09-16T11:45:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442892+0000", 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028589564000000001 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442930+0000", 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330999999999999e-05 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442977+0000", 2024-09-16T11:45:10.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7874000000000003e-05 2024-09-16T11:45:10.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.432 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.432 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41956 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.432 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:45:10.432 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987975399, 2024-09-16T11:45:10.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028766896, 2024-09-16T11:45:10.432 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.432 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.432 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.433 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41956 2024-09-16T11:45:10.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.433 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:45:10.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414326+0000", 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9900000000000009e-07 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414330+0000", 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2339999999999996e-06 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414341+0000", 2024-09-16T11:45:10.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1358999999999999e-05 2024-09-16T11:45:10.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414347+0000", 2024-09-16T11:45:10.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8100000000000003e-06 2024-09-16T11:45:10.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443002+0000", 2024-09-16T11:45:10.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654331000000002 2024-09-16T11:45:10.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443040+0000", 2024-09-16T11:45:10.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8262000000000001e-05 2024-09-16T11:45:10.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443092+0000", 2024-09-16T11:45:10.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1901000000000003e-05 2024-09-16T11:45:10.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.436 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41957 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987947980000001, 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028848318000000001, 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41957 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.437 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414353+0000", 2024-09-16T11:45:10.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414355+0000", 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3489999999999999e-06 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414364+0000", 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1449999999999997e-06 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414368+0000", 2024-09-16T11:45:10.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8940000000000003e-06 2024-09-16T11:45:10.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443115+0000", 2024-09-16T11:45:10.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028746532000000002 2024-09-16T11:45:10.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443152+0000", 2024-09-16T11:45:10.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6814999999999999e-05 2024-09-16T11:45:10.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443201+0000", 2024-09-16T11:45:10.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9069000000000001e-05 2024-09-16T11:45:10.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.441 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.441 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41958 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.441 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:45:10.441 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987924198000002, 2024-09-16T11:45:10.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028939856, 2024-09-16T11:45:10.441 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41958 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414377+0000", 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4099999999999999e-07 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414379+0000", 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.418e-06 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414385+0000", 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999999e-06 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414389+0000", 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7740000000000002e-06 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443223+0000", 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028833919 2024-09-16T11:45:10.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443263+0000", 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0148999999999998e-05 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443316+0000", 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2636000000000002e-05 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41959 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987907272999998, 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029040309, 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41959 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414395+0000", 2024-09-16T11:45:10.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7579999999999999e-06 2024-09-16T11:45:10.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414401+0000", 2024-09-16T11:45:10.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5330000000000002e-06 2024-09-16T11:45:10.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414404+0000", 2024-09-16T11:45:10.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.529e-06 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443339+0000", 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028934933999999999 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443383+0000", 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3832000000000003e-05 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443433+0000", 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0309000000000002e-05 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41960 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987892438999999, 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029138134, 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.450 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.451 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41960 2024-09-16T11:45:10.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.451 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:10.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:10.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:10.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-07 2024-09-16T11:45:10.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414410+0000", 2024-09-16T11:45:10.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:45:10.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414417+0000", 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414420+0000", 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443457+0000", 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029037065000000001 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443495+0000", 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7555000000000002e-05 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443546+0000", 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1422000000000003e-05 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41961 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.454 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987876458000002, 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029230973, 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41961 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4499999999999998e-07 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414432+0000", 2024-09-16T11:45:10.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.413e-06 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414439+0000", 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4769999999999997e-06 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414443+0000", 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8859999999999997e-06 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443569+0000", 2024-09-16T11:45:10.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029126124 2024-09-16T11:45:10.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443606+0000", 2024-09-16T11:45:10.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6588999999999998e-05 2024-09-16T11:45:10.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443655+0000", 2024-09-16T11:45:10.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9138999999999998e-05 2024-09-16T11:45:10.458 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.458 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41962 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987853491000003, 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029324505000000001, 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41962 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.459 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:10.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:10.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:45:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414449+0000", 2024-09-16T11:45:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0279999999999999e-06 2024-09-16T11:45:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414456+0000", 2024-09-16T11:45:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3339999999999998e-06 2024-09-16T11:45:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414459+0000", 2024-09-16T11:45:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.382e-06 2024-09-16T11:45:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443678+0000", 2024-09-16T11:45:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029218958999999999 2024-09-16T11:45:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443719+0000", 2024-09-16T11:45:10.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0722000000000001e-05 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443771+0000", 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2661000000000003e-05 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41963 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987837587999998, 2024-09-16T11:45:10.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029416557999999999, 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41963 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:10.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:10.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-16T11:45:10.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414465+0000", 2024-09-16T11:45:10.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:45:10.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414470+0000", 2024-09-16T11:45:10.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4750000000000001e-06 2024-09-16T11:45:10.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414474+0000", 2024-09-16T11:45:10.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3720000000000001e-06 2024-09-16T11:45:10.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443795+0000", 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029320821 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443830+0000", 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5463000000000002e-05 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443879+0000", 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9147000000000002e-05 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.467 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41964 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987822700000002, 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029521202999999999, 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41964 2024-09-16T11:45:10.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.469 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:10.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:10.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:10.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:45:10.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414480+0000", 2024-09-16T11:45:10.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:45:10.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414486+0000", 2024-09-16T11:45:10.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:45:10.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414489+0000", 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2770000000000001e-06 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443903+0000", 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029414109000000001 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443944+0000", 2024-09-16T11:45:10.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0992999999999999e-05 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443999+0000", 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4632000000000001e-05 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41965 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987807738000001, 2024-09-16T11:45:10.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029611293, 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41965 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414495+0000", 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.88e-06 2024-09-16T11:45:10.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414501+0000", 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2489999999999998e-06 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414504+0000", 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3139999999999999e-06 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444023+0000", 2024-09-16T11:45:10.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029518564000000001 2024-09-16T11:45:10.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444058+0000", 2024-09-16T11:45:10.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5196e-05 2024-09-16T11:45:10.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444104+0000", 2024-09-16T11:45:10.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5732999999999997e-05 2024-09-16T11:45:10.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.476 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41966 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987791532000003, 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029707678000000001, 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41966 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.477 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:10.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:10.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414514+0000", 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414518+0000", 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3839999999999999e-06 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414521+0000", 2024-09-16T11:45:10.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0400000000000001e-06 2024-09-16T11:45:10.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444128+0000", 2024-09-16T11:45:10.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029606612000000001 2024-09-16T11:45:10.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444167+0000", 2024-09-16T11:45:10.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9418000000000001e-05 2024-09-16T11:45:10.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444216+0000", 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9354999999999998e-05 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41967 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987775722999999, 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029796916, 2024-09-16T11:45:10.481 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.482 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.482 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41967 2024-09-16T11:45:10.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.482 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:10.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:10.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:10.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6699999999999999e-07 2024-09-16T11:45:10.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414527+0000", 2024-09-16T11:45:10.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7060000000000001e-06 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414533+0000", 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2140000000000001e-06 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414536+0000", 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444239+0000", 2024-09-16T11:45:10.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029702698999999999 2024-09-16T11:45:10.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444273+0000", 2024-09-16T11:45:10.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4229e-05 2024-09-16T11:45:10.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444321+0000", 2024-09-16T11:45:10.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8477999999999999e-05 2024-09-16T11:45:10.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.485 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41968 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987760825999999, 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029902765000000001, 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41968 2024-09-16T11:45:10.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414540+0000", 2024-09-16T11:45:10.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:45:10.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414542+0000", 2024-09-16T11:45:10.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:45:10.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414547+0000", 2024-09-16T11:45:10.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4639999999999997e-06 2024-09-16T11:45:10.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414551+0000", 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6279999999999998e-06 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444346+0000", 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029794885 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444389+0000", 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3442000000000002e-05 2024-09-16T11:45:10.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444442+0000", 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3173000000000001e-05 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41969 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.987745914000001, 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029997612999999999, 2024-09-16T11:45:10.490 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41969 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414555+0000", 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414556+0000", 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.725e-06 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414563+0000", 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2530000000000001e-06 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414566+0000", 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4429999999999999e-06 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444463+0000", 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029897271 2024-09-16T11:45:10.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444503+0000", 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9782e-05 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444552+0000", 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8789000000000003e-05 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.494 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:10.496 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:45:10.496 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:10.494+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:10.596 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:10.595+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:10.645 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:10.645 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:10.646 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:10.646 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:10.651 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T11:45:10.651 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.3 2024-09-16T11:45:10.652 INFO:tasks.thrashosds.thrasher:remote for osd 3 is ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T11:45:10.652 DEBUG:teuthology.orchestra.run.smithi070:> type ceph-objectstore-tool 2024-09-16T11:45:10.655 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:45:10.667 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T11:45:10.667 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op list-pgs 2024-09-16T11:45:10.701 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:10.699+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:10.754 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:10.754 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:10.754 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:10.754 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:10.754 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:10.762 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:45:10.797 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:10.796+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:10.899 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:10.897+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:10.901 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:10.901 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:10.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:10.902 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:10.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.902 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:10.902 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:45:10.902 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 117.978885475, 2024-09-16T11:45:10.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071340320000000002, 2024-09-16T11:45:10.902 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.902 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:10.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:10.902 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-16T11:45:10.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.903 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:45:10.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:45:10.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920024+0000", 2024-09-16T11:45:10.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2499999999999999e-07 2024-09-16T11:45:10.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920036+0000", 2024-09-16T11:45:10.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2515e-05 2024-09-16T11:45:10.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920040+0000", 2024-09-16T11:45:10.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:45:10.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920044+0000", 2024-09-16T11:45:10.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:45:10.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923562+0000", 2024-09-16T11:45:10.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035177670000000002 2024-09-16T11:45:10.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923623+0000", 2024-09-16T11:45:10.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1254999999999995e-05 2024-09-16T11:45:10.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:10.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.923717+0000", 2024-09-16T11:45:10.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4006999999999996e-05 2024-09-16T11:45:10.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:10.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924854+0000", 2024-09-16T11:45:10.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001136859 2024-09-16T11:45:10.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:10.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927108+0000", 2024-09-16T11:45:10.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0022539270000000002 2024-09-16T11:45:10.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927136+0000", 2024-09-16T11:45:10.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8096e-05 2024-09-16T11:45:10.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927158+0000", 2024-09-16T11:45:10.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2008999999999999e-05 2024-09-16T11:45:10.908 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.908 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.908 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.908 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:10.908 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:45:10.908 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 117.978839783, 2024-09-16T11:45:10.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:45:10.908 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.908 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:10.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:10.909 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:45:10.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.909 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:45:10.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:45:10.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:45:10.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:45:10.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:45:10.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:45:10.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:45:10.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:45:10.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:45:10.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:45:10.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:45:10.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:45:10.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:45:10.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:45:10.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:10.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:45:10.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:45:10.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:45:10.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:45:10.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.914 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.914 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 117.978782853, 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:45:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:45:10.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:45:10.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:45:10.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:45:10.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:45:10.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:45:10.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:45:10.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:45:10.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:10.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:45:10.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:45:10.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:10.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:45:10.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:45:10.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:10.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:45:10.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:45:10.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:45:10.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:45:10.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:45:10.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:45:10.921 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.921 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.921 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.922 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:10.922 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:10.922 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 117.978757485, 2024-09-16T11:45:10.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:45:10.922 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.922 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:10.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:10.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:45:10.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:45:10.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:45:10.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:45:10.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:45:10.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:45:10.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:45:10.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:45:10.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:45:10.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:45:10.928 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.929 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.929 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:10.929 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:10.929 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 117.97870551699999, 2024-09-16T11:45:10.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:45:10.929 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.929 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:10.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:10.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:45:10.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:45:10.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:45:10.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:45:10.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:45:10.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:45:10.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:10.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:45:10.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:45:10.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:10.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:45:10.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:45:10.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:10.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:45:10.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:45:10.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:45:10.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:45:10.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:45:10.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:45:10.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.935 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.935 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:10.935 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:10.935 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 117.978624407, 2024-09-16T11:45:10.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:45:10.936 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.936 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:10.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:10.936 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:45:10.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.936 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:10.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:10.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:10.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:45:10.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:45:10.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:45:10.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:45:10.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:45:10.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:45:10.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:45:10.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:45:10.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:45:10.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:45:10.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:45:10.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:10.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:45:10.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:45:10.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:10.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:45:10.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:45:10.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:10.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:45:10.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:45:10.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:45:10.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:45:10.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:45:10.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:45:10.944 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.944 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.944 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.945 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:10.945 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:10.945 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 117.978598429, 2024-09-16T11:45:10.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:45:10.945 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.946 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:10.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:10.946 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:45:10.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.946 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:10.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:10.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:10.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:45:10.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:45:10.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:45:10.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:45:10.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:45:10.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:45:10.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:45:10.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:45:10.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:45:10.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:10.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:45:10.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:45:10.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:45:10.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:45:10.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:45:10.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:45:10.951 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.952 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.952 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.952 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:10.952 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:45:10.952 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 117.561079804, 2024-09-16T11:45:10.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:45:10.952 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.952 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:10.953 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:10.953 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:45:10.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.953 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:45:10.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:45:10.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:45:10.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:45:10.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:45:10.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:45:10.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:45:10.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:45:10.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:45:10.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:45:10.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:45:10.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:45:10.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:45:10.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:45:10.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:10.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:45:10.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:45:10.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:10.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:45:10.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:45:10.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:10.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:45:10.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:45:10.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:45:10.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:45:10.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:45:10.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:45:10.959 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.959 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.959 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.959 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:10.960 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:45:10.960 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 117.561057873, 2024-09-16T11:45:10.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:45:10.960 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.960 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.960 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.960 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:10.960 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:10.960 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:45:10.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.960 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:45:10.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:45:10.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:45:10.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:45:10.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:45:10.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:45:10.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:45:10.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:45:10.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:45:10.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:45:10.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:45:10.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:45:10.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:45:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:45:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:45:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:45:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:45:10.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:45:10.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:10.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:45:10.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:45:10.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:45:10.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:45:10.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:45:10.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:45:10.966 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.966 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.966 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.966 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:10.966 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:45:10.966 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 117.560980985, 2024-09-16T11:45:10.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:45:10.966 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.967 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.967 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:10.967 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:10.967 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:10.967 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:45:10.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.967 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:45:10.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:45:10.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:45:10.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:45:10.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:45:10.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:45:10.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:45:10.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:45:10.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:45:10.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:45:10.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:45:10.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:45:10.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:45:10.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:45:10.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:10.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:45:10.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:45:10.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:10.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:45:10.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:45:10.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:10.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:45:10.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:45:10.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:45:10.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:45:10.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:45:10.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:45:10.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.972 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.973 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28187 4.17 e52/28 4:ee916bd9:::rbd_data.11c1ce740b46.0000000000000000:head v 52'195, mlcod=52'193)", 2024-09-16T11:45:10.973 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:45:10.973 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 70.389561599000004, 2024-09-16T11:45:10.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075854429999999999, 2024-09-16T11:45:10.973 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.973 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.973 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:45:10.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:45:10.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509349+0000", 2024-09-16T11:45:10.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2300000000000001e-06 2024-09-16T11:45:10.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513315+0000", 2024-09-16T11:45:10.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039660590000000001 2024-09-16T11:45:10.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513328+0000", 2024-09-16T11:45:10.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.308e-05 2024-09-16T11:45:10.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513334+0000", 2024-09-16T11:45:10.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:45:10.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513385+0000", 2024-09-16T11:45:10.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1347000000000002e-05 2024-09-16T11:45:10.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513407+0000", 2024-09-16T11:45:10.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1852999999999999e-05 2024-09-16T11:45:10.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516882+0000", 2024-09-16T11:45:10.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034749690000000001 2024-09-16T11:45:10.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516933+0000", 2024-09-16T11:45:10.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0952999999999998e-05 2024-09-16T11:45:10.977 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.978 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.978 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.978 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28959 4.8 e53/28 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head v 53'408, mlcod=53'406)", 2024-09-16T11:45:10.978 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:45:10.978 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.867412543, 2024-09-16T11:45:10.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088740420000000004, 2024-09-16T11:45:10.978 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.978 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.978 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:45:10.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:45:10.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031498+0000", 2024-09-16T11:45:10.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.652e-06 2024-09-16T11:45:10.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033515+0000", 2024-09-16T11:45:10.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020165930000000001 2024-09-16T11:45:10.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033539+0000", 2024-09-16T11:45:10.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4295999999999999e-05 2024-09-16T11:45:10.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033548+0000", 2024-09-16T11:45:10.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1239999999999994e-06 2024-09-16T11:45:10.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033610+0000", 2024-09-16T11:45:10.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1371999999999994e-05 2024-09-16T11:45:10.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033637+0000", 2024-09-16T11:45:10.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7098999999999999e-05 2024-09-16T11:45:10.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040341+0000", 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067045819999999997 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040371+0000", 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9323999999999999e-05 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.982 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32303 4.8 e53/28 4:1436ac82:::rbd_data.11c1cab081b3.0000000000000022:head v 53'417, mlcod=53'415)", 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.396986569000006, 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085565160000000001, 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:10.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:10.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-09-16T11:45:10.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501930+0000", 2024-09-16T11:45:10.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7649999999999996e-06 2024-09-16T11:45:10.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501946+0000", 2024-09-16T11:45:10.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5301000000000001e-05 2024-09-16T11:45:10.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501952+0000", 2024-09-16T11:45:10.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:45:10.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502100+0000", 2024-09-16T11:45:10.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014847600000000001 2024-09-16T11:45:10.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502120+0000", 2024-09-16T11:45:10.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9303000000000001e-05 2024-09-16T11:45:10.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510436+0000", 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083166420000000008 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510479+0000", 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2979000000000003e-05 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32306 4.1e e53/28 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head v 53'124, mlcod=53'122)", 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:10.987 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.396920574999996, 2024-09-16T11:45:10.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084990900000000008, 2024-09-16T11:45:10.988 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.988 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.988 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:10.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:45:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501994+0000", 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502002+0000", 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3839999999999997e-06 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502007+0000", 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6290000000000001e-06 2024-09-16T11:45:10.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502454+0000", 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00044736800000000002 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502471+0000", 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6648999999999999e-05 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510455+0000", 2024-09-16T11:45:10.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0079841909999999999 2024-09-16T11:45:10.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510488+0000", 2024-09-16T11:45:10.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2948999999999999e-05 2024-09-16T11:45:10.992 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.992 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.992 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.992 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32307 4.8 e53/28 4:11c1c37d:::rbd_data.11c1cab081b3.0000000000000026:head v 53'418, mlcod=53'415)", 2024-09-16T11:45:10.992 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:10.992 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.396165240000002, 2024-09-16T11:45:10.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00776212, 2024-09-16T11:45:10.993 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.993 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.993 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:10.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:10.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502750+0000", 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502760+0000", 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502767+0000", 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4219999999999997e-06 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502802+0000", 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5089999999999998e-05 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:10.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502816+0000", 2024-09-16T11:45:10.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4092e-05 2024-09-16T11:45:10.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:10.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510483+0000", 2024-09-16T11:45:10.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00766704 2024-09-16T11:45:10.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:10.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510506+0000", 2024-09-16T11:45:10.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3195999999999999e-05 2024-09-16T11:45:10.996 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.997 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:10.997 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:10.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.997 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33583 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'416, mlcod=53'411)", 2024-09-16T11:45:10.997 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:45:10.997 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.688206831000002, 2024-09-16T11:45:10.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078132459999999994, 2024-09-16T11:45:10.997 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:10.997 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:10.997 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:10.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:10.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:45:10.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:10.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:45:10.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:10.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:10.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210703+0000", 2024-09-16T11:45:10.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:45:10.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:10.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210706+0000", 2024-09-16T11:45:10.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-06 2024-09-16T11:45:10.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:10.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210712+0000", 2024-09-16T11:45:10.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:45:10.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:10.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:10.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:10.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210715+0000", 2024-09-16T11:45:11.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:45:11.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:11.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210736+0000", 2024-09-16T11:45:11.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0293999999999999e-05 2024-09-16T11:45:11.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:11.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210746+0000", 2024-09-16T11:45:11.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0178e-05 2024-09-16T11:45:11.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:11.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218477+0000", 2024-09-16T11:45:11.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007731501 2024-09-16T11:45:11.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:11.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218516+0000", 2024-09-16T11:45:11.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8148000000000001e-05 2024-09-16T11:45:11.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:11.001 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:11.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:11.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33590 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'419, mlcod=53'415)", 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.677148689999996, 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082499879999999998, 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:11.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:45:11.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:11.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:11.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221761+0000", 2024-09-16T11:45:11.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-16T11:45:11.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:11.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221765+0000", 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1590000000000003e-06 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221774+0000", 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7490000000000004e-06 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221778+0000", 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0419999999999996e-06 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:11.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221800+0000", 2024-09-16T11:45:11.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1926999999999999e-05 2024-09-16T11:45:11.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:11.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221810+0000", 2024-09-16T11:45:11.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0079e-05 2024-09-16T11:45:11.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:11.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.229980+0000", 2024-09-16T11:45:11.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081698469999999992 2024-09-16T11:45:11.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230010+0000", 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0746e-05 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33591 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'420, mlcod=53'415)", 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.676587205999994, 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007702576, 2024-09-16T11:45:11.006 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222326+0000", 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222337+0000", 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1008e-05 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:11.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222342+0000", 2024-09-16T11:45:11.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0050000000000004e-06 2024-09-16T11:45:11.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:11.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222366+0000", 2024-09-16T11:45:11.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3696000000000001e-05 2024-09-16T11:45:11.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:11.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222385+0000", 2024-09-16T11:45:11.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-16T11:45:11.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230012+0000", 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076270940000000001 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230025+0000", 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2112e-05 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.010 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:55282 4.1e e73/28 4:785299e2:::rbd_data.11c11a49c2da.0000000000000001:head v 73'324, mlcod=73'322)", 2024-09-16T11:45:11.011 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:45:11.011 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.659890687000001, 2024-09-16T11:45:11.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071431890000000003, 2024-09-16T11:45:11.011 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:11.011 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:11.011 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:11.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:11.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:45:11.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:11.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239020+0000", 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.049e-06 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244238+0000", 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052183489999999997 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244252+0000", 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4246999999999999e-05 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244275+0000", 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3093999999999999e-05 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244314+0000", 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8495000000000003e-05 2024-09-16T11:45:11.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244330+0000", 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5917e-05 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246121+0000", 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017909239999999999 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246162+0000", 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1114e-05 2024-09-16T11:45:11.014 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:11.015 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:11.015 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:11.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.015 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:61651 4.11 e83/28 4:8f4a51c5:::rbd_data.11c175f8cab7.0000000000000014:head v 83'694, mlcod=81'692)", 2024-09-16T11:45:11.015 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:45:11.015 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.285471139999999, 2024-09-16T11:45:11.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.009466482, 2024-09-16T11:45:11.015 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:11.015 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:11.015 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:11.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:11.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:45:11.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:11.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:11.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:45:11.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:11.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:11.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613439+0000", 2024-09-16T11:45:11.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8400000000000003e-07 2024-09-16T11:45:11.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:11.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618867+0000", 2024-09-16T11:45:11.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054276059999999998 2024-09-16T11:45:11.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:11.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618883+0000", 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6265000000000001e-05 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618890+0000", 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2980000000000003e-06 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618950+0000", 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0025999999999998e-05 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:11.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618971+0000", 2024-09-16T11:45:11.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0458999999999998e-05 2024-09-16T11:45:11.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:11.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622869+0000", 2024-09-16T11:45:11.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038983450000000001 2024-09-16T11:45:11.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:11.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:11.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:11.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622905+0000", 2024-09-16T11:45:11.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599000000000003e-05 2024-09-16T11:45:11.019 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:11.019 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:11.020 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:11.020 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:11.020 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:11.020 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:11.021 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:11.018+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:11.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:45:11.071 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:45:11.122 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:11.119+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:11.123 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:45:11.123 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:11.132 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:45:11.200 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:45:11.200 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:45:11.222 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:45:11.223 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:11.220+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:11.223 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:45:11.234 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:11.234 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:11.237 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:45:11.238 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:45:11.238 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T11:45:11.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:45:11.254 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:45:11.255 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:45:11.271 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:45:11.271 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:45:11.301 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:45:11.301 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:45:11.323 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:11.321+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:11.330 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:45:11.330 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:45:11.334 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:45:11.335 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:45:11.363 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:11.363 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:11.371 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:45:11.424 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:11.422+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:11.521 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:11.521 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:11.525 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:11.523+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:11.530 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:45:11.560 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:45:11.626 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:11.624+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:11.634 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:11.634 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:11.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:45:11.727 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:11.726+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:11.782 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:11.782 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:11.791 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:45:11.828 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:11.826+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:11.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:45:11.850+0000 7f32e5615200 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-16T11:45:11.890 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:11.891 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:11.899 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:45:11.929 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:11.927+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:11.996 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:11.996 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:12.005 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:45:12.048 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:12.046+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:12.104 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:12.104 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:12.112 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:45:12.156 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:12.154+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:12.221 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:12.221 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:12.230 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:45:12.271 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:12.268+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:12.332 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:12.330+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:12.369 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:12.369 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:12.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:45:12.433 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:12.431+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:12.514 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:12.515 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:12.523 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:45:12.556 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T11:45:12.566 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:12.564+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:12.630 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:12.630 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:12.635 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:12.633+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:12.639 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:45:12.737 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:12.734+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:12.778 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:12.778 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:12.786 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:45:12.837 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:12.835+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:12.934 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:45:12.977 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:12.975+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:13.032 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:13.032 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:13.039 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:13.036+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:13.041 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:45:13.139 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:13.137+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:13.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-16T11:45:13.179 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:13.179 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:13.188 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:45:13.241 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:13.239+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:13.324 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:13.325 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:13.333 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:45:13.336 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:45:13.377 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:13.375+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:13.431 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:13.431 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:13.440 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:45:13.450 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:45:13.451 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:45:13.466 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:45:13.466 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:45:13.480 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:45:13.481 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:45:13.481 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T11:45:13.482 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:13.479+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:13.496 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:45:13.496 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:45:13.511 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:45:13.511 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:45:13.526 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:45:13.526 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:45:13.541 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:45:13.542 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:45:13.543 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:13.540+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:13.544 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:45:13.545 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:45:13.582 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:13.582 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:13.591 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:45:13.676 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:13.674+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:13.734 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:13.734 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:13.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:13.734 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:13.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.734 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:13.734 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:45:13.734 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.811062632, 2024-09-16T11:45:13.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:45:13.735 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.735 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.735 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:13.735 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:13.735 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:13.735 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:45:13.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.735 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:45:13.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:45:13.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:45:13.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:45:13.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:45:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:45:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:45:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:45:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:45:13.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:13.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:45:13.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:45:13.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:45:13.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:45:13.740 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.740 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.740 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.741 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:13.741 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:45:13.741 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.811005702, 2024-09-16T11:45:13.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:45:13.741 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.741 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:13.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:13.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:13.741 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:45:13.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.742 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:45:13.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:45:13.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:45:13.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:45:13.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:45:13.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:45:13.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:45:13.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:45:13.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:45:13.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:45:13.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:45:13.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:45:13.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:45:13.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:45:13.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:13.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:45:13.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:45:13.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:13.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:45:13.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:45:13.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:13.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:45:13.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:45:13.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:45:13.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:45:13.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:45:13.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:45:13.747 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.747 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.747 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.747 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:13.747 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:13.747 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.81098033400001, 2024-09-16T11:45:13.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:45:13.748 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.748 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.748 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:13.748 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:13.748 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:13.748 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:45:13.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.748 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:13.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:13.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:13.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:45:13.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:45:13.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:45:13.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:45:13.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:45:13.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:45:13.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:45:13.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:45:13.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:45:13.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:45:13.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:45:13.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:13.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:45:13.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:45:13.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:13.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:45:13.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:45:13.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:13.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:45:13.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:45:13.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:45:13.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:45:13.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:45:13.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:45:13.753 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.754 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.754 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.754 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:13.754 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:13.754 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.810928366, 2024-09-16T11:45:13.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:45:13.754 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.754 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:13.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:13.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:45:13.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:45:13.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:45:13.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:45:13.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:45:13.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:45:13.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:45:13.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:45:13.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:45:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:45:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:45:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:45:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:45:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:45:13.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:45:13.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:45:13.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:45:13.760 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.760 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.760 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.760 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.810847256, 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:45:13.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:45:13.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:45:13.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:45:13.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:13.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:45:13.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:45:13.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:13.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:45:13.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:45:13.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:13.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:45:13.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.81082127800001, 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:13.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:13.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:13.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:45:13.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:45:13.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:45:13.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:45:13.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:45:13.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:45:13.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:13.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:45:13.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:45:13.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:13.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:45:13.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:45:13.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:13.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:45:13.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:45:13.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:45:13.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:45:13.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:45:13.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:45:13.772 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.772 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.773 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.773 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:13.773 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:45:13.773 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.39330265300001, 2024-09-16T11:45:13.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:45:13.773 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.773 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.773 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:13.773 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:13.773 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:13.773 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:45:13.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:45:13.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:45:13.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:45:13.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:45:13.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:45:13.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:45:13.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:45:13.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:45:13.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:45:13.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:13.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:45:13.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:45:13.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:45:13.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:45:13.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:45:13.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:45:13.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.779 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.779 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:13.779 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:45:13.779 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.393280722, 2024-09-16T11:45:13.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:45:13.779 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.779 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.779 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:13.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:13.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:13.780 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:45:13.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.780 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:45:13.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:45:13.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:45:13.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:45:13.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:45:13.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:45:13.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:45:13.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:45:13.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:45:13.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:13.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:45:13.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:45:13.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:13.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:45:13.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:45:13.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:13.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:45:13.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:45:13.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:45:13.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:45:13.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:45:13.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:45:13.785 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.785 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.785 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.785 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:13.786 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:45:13.786 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.393203834, 2024-09-16T11:45:13.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:45:13.786 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.786 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.786 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:13.786 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:13.786 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:13.786 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:45:13.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.786 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:45:13.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:45:13.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:45:13.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:45:13.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:45:13.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:45:13.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:45:13.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:45:13.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:45:13.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:45:13.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:13.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:45:13.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:45:13.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:13.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:45:13.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:45:13.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:13.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:45:13.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:45:13.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:45:13.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:45:13.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:45:13.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:45:13.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.795 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.796 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28187 4.17 e52/28 4:ee916bd9:::rbd_data.11c1ce740b46.0000000000000000:head v 52'195, mlcod=52'193)", 2024-09-16T11:45:13.796 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:45:13.796 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.221784447999994, 2024-09-16T11:45:13.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075854429999999999, 2024-09-16T11:45:13.796 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.796 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.796 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:45:13.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:45:13.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509349+0000", 2024-09-16T11:45:13.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2300000000000001e-06 2024-09-16T11:45:13.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513315+0000", 2024-09-16T11:45:13.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039660590000000001 2024-09-16T11:45:13.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513328+0000", 2024-09-16T11:45:13.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.308e-05 2024-09-16T11:45:13.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513334+0000", 2024-09-16T11:45:13.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:45:13.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513385+0000", 2024-09-16T11:45:13.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1347000000000002e-05 2024-09-16T11:45:13.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513407+0000", 2024-09-16T11:45:13.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1852999999999999e-05 2024-09-16T11:45:13.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516882+0000", 2024-09-16T11:45:13.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034749690000000001 2024-09-16T11:45:13.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516933+0000", 2024-09-16T11:45:13.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0952999999999998e-05 2024-09-16T11:45:13.805 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.805 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.805 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.806 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28959 4.8 e53/28 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head v 53'408, mlcod=53'406)", 2024-09-16T11:45:13.806 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:45:13.806 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.699635392000005, 2024-09-16T11:45:13.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088740420000000004, 2024-09-16T11:45:13.806 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.806 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.806 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:45:13.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:45:13.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031498+0000", 2024-09-16T11:45:13.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.652e-06 2024-09-16T11:45:13.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033515+0000", 2024-09-16T11:45:13.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020165930000000001 2024-09-16T11:45:13.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033539+0000", 2024-09-16T11:45:13.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4295999999999999e-05 2024-09-16T11:45:13.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033548+0000", 2024-09-16T11:45:13.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1239999999999994e-06 2024-09-16T11:45:13.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033610+0000", 2024-09-16T11:45:13.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1371999999999994e-05 2024-09-16T11:45:13.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033637+0000", 2024-09-16T11:45:13.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7098999999999999e-05 2024-09-16T11:45:13.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040341+0000", 2024-09-16T11:45:13.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067045819999999997 2024-09-16T11:45:13.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040371+0000", 2024-09-16T11:45:13.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9323999999999999e-05 2024-09-16T11:45:13.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.810 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32303 4.8 e53/28 4:1436ac82:::rbd_data.11c1cab081b3.0000000000000022:head v 53'417, mlcod=53'415)", 2024-09-16T11:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.229209417999996, 2024-09-16T11:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085565160000000001, 2024-09-16T11:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:13.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:13.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-09-16T11:45:13.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501930+0000", 2024-09-16T11:45:13.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7649999999999996e-06 2024-09-16T11:45:13.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501946+0000", 2024-09-16T11:45:13.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5301000000000001e-05 2024-09-16T11:45:13.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501952+0000", 2024-09-16T11:45:13.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:45:13.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502100+0000", 2024-09-16T11:45:13.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014847600000000001 2024-09-16T11:45:13.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502120+0000", 2024-09-16T11:45:13.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9303000000000001e-05 2024-09-16T11:45:13.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510436+0000", 2024-09-16T11:45:13.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083166420000000008 2024-09-16T11:45:13.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510479+0000", 2024-09-16T11:45:13.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2979000000000003e-05 2024-09-16T11:45:13.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.815 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.815 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32306 4.1e e53/28 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head v 53'124, mlcod=53'122)", 2024-09-16T11:45:13.816 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:13.816 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.229143424, 2024-09-16T11:45:13.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084990900000000008, 2024-09-16T11:45:13.816 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.816 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.816 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:13.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:13.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:13.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:45:13.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501994+0000", 2024-09-16T11:45:13.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:45:13.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502002+0000", 2024-09-16T11:45:13.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3839999999999997e-06 2024-09-16T11:45:13.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502007+0000", 2024-09-16T11:45:13.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6290000000000001e-06 2024-09-16T11:45:13.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502454+0000", 2024-09-16T11:45:13.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00044736800000000002 2024-09-16T11:45:13.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502471+0000", 2024-09-16T11:45:13.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6648999999999999e-05 2024-09-16T11:45:13.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510455+0000", 2024-09-16T11:45:13.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0079841909999999999 2024-09-16T11:45:13.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510488+0000", 2024-09-16T11:45:13.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2948999999999999e-05 2024-09-16T11:45:13.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.820 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.820 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32307 4.8 e53/28 4:11c1c37d:::rbd_data.11c1cab081b3.0000000000000026:head v 53'418, mlcod=53'415)", 2024-09-16T11:45:13.820 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:13.821 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.228388089000006, 2024-09-16T11:45:13.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00776212, 2024-09-16T11:45:13.821 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.821 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.821 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:13.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:13.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:13.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:45:13.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502750+0000", 2024-09-16T11:45:13.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:45:13.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502760+0000", 2024-09-16T11:45:13.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:45:13.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502767+0000", 2024-09-16T11:45:13.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4219999999999997e-06 2024-09-16T11:45:13.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502802+0000", 2024-09-16T11:45:13.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5089999999999998e-05 2024-09-16T11:45:13.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502816+0000", 2024-09-16T11:45:13.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4092e-05 2024-09-16T11:45:13.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510483+0000", 2024-09-16T11:45:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00766704 2024-09-16T11:45:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510506+0000", 2024-09-16T11:45:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3195999999999999e-05 2024-09-16T11:45:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33583 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'416, mlcod=53'411)", 2024-09-16T11:45:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:45:13.826 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.520429680000007, 2024-09-16T11:45:13.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078132459999999994, 2024-09-16T11:45:13.826 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.826 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.826 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:45:13.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:45:13.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210703+0000", 2024-09-16T11:45:13.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:45:13.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210706+0000", 2024-09-16T11:45:13.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-06 2024-09-16T11:45:13.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210712+0000", 2024-09-16T11:45:13.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:45:13.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210715+0000", 2024-09-16T11:45:13.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:45:13.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210736+0000", 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0293999999999999e-05 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210746+0000", 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0178e-05 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218477+0000", 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007731501 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218516+0000", 2024-09-16T11:45:13.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8148000000000001e-05 2024-09-16T11:45:13.830 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.830 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.830 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.830 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33590 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'419, mlcod=53'415)", 2024-09-16T11:45:13.830 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:45:13.830 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.509371539, 2024-09-16T11:45:13.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082499879999999998, 2024-09-16T11:45:13.830 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.830 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.831 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:45:13.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:45:13.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221761+0000", 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221765+0000", 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1590000000000003e-06 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221774+0000", 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7490000000000004e-06 2024-09-16T11:45:13.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221778+0000", 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0419999999999996e-06 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221800+0000", 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1926999999999999e-05 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221810+0000", 2024-09-16T11:45:13.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0079e-05 2024-09-16T11:45:13.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.229980+0000", 2024-09-16T11:45:13.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081698469999999992 2024-09-16T11:45:13.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230010+0000", 2024-09-16T11:45:13.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0746e-05 2024-09-16T11:45:13.834 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.834 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.834 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33591 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'420, mlcod=53'415)", 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.508810054999998, 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007702576, 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:13.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:13.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:45:13.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222326+0000", 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222337+0000", 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1008e-05 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222342+0000", 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0050000000000004e-06 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222366+0000", 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3696000000000001e-05 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222385+0000", 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230012+0000", 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076270940000000001 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230025+0000", 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2112e-05 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:55282 4.1e e73/28 4:785299e2:::rbd_data.11c11a49c2da.0000000000000001:head v 73'324, mlcod=73'322)", 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.492113535999998, 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071431890000000003, 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:45:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:45:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239020+0000", 2024-09-16T11:45:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.049e-06 2024-09-16T11:45:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244238+0000", 2024-09-16T11:45:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052183489999999997 2024-09-16T11:45:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244252+0000", 2024-09-16T11:45:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4246999999999999e-05 2024-09-16T11:45:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244275+0000", 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3093999999999999e-05 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244314+0000", 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8495000000000003e-05 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244330+0000", 2024-09-16T11:45:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5917e-05 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246121+0000", 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017909239999999999 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246162+0000", 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1114e-05 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:61651 4.11 e83/28 4:8f4a51c5:::rbd_data.11c175f8cab7.0000000000000014:head v 83'694, mlcod=81'692)", 2024-09-16T11:45:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:45:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 39.117693989000003, 2024-09-16T11:45:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.009466482, 2024-09-16T11:45:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:45:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:45:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613439+0000", 2024-09-16T11:45:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8400000000000003e-07 2024-09-16T11:45:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618867+0000", 2024-09-16T11:45:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054276059999999998 2024-09-16T11:45:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618883+0000", 2024-09-16T11:45:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6265000000000001e-05 2024-09-16T11:45:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618890+0000", 2024-09-16T11:45:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2980000000000003e-06 2024-09-16T11:45:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618950+0000", 2024-09-16T11:45:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0025999999999998e-05 2024-09-16T11:45:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618971+0000", 2024-09-16T11:45:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0458999999999998e-05 2024-09-16T11:45:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622869+0000", 2024-09-16T11:45:13.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038983450000000001 2024-09-16T11:45:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622905+0000", 2024-09-16T11:45:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599000000000003e-05 2024-09-16T11:45:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:95694 4.17 4:ec7bbff8:::rbd_data.11c18a2497a7.0000000000000008:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-16T11:45:13.848 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:45:13.626080+0000", 2024-09-16T11:45:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.105051722, 2024-09-16T11:45:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0077344190000000002, 2024-09-16T11:45:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 95694 2024-09-16T11:45:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:13.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626080+0000", 2024-09-16T11:45:13.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:13.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626080+0000", 2024-09-16T11:45:13.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:13.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:13.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626083+0000", 2024-09-16T11:45:13.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3180000000000001e-06 2024-09-16T11:45:13.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626089+0000", 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626105+0000", 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5903999999999999e-05 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626112+0000", 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4649999999999997e-06 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626162+0000", 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0238999999999998e-05 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626167+0000", 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8019999999999999e-06 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.628097+0000", 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019296719999999999 2024-09-16T11:45:13.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.628165+0000", 2024-09-16T11:45:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8313999999999997e-05 2024-09-16T11:45:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.633698+0000", 2024-09-16T11:45:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0055331410000000001 2024-09-16T11:45:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:13.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.633742+0000", 2024-09-16T11:45:13.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3408999999999997e-05 2024-09-16T11:45:13.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:13.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:13.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:13.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.633815+0000", 2024-09-16T11:45:13.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2935999999999994e-05 2024-09-16T11:45:13.854 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.854 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.854 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.854 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:13.854 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:13.854 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:13.856 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:13.854+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:13.856 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:45:13.955 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:13.955 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:45:13.955 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:13.956 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:13.954+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:13.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:45:14.058 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:14.056+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:14.103 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:14.103 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:14.103 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:45:14.103 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:45:14.103 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:14.111 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:45:14.195 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:14.193+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:14.251 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:14.251 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:45:14.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:45:14.252 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:45:14.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.252 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89607 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1327104~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.252 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:45:14.252 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.321568736000003, 2024-09-16T11:45:14.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.033363680999999999, 2024-09-16T11:45:14.252 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.252 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.252 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.252 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89607 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.927229+0000", 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929100+0000", 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001870776 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929104+0000", 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1160000000000001e-06 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929118+0000", 2024-09-16T11:45:14.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3891e-05 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929126+0000", 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6590000000000007e-06 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960508+0000", 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031381920000000001 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960548+0000", 2024-09-16T11:45:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0734e-05 2024-09-16T11:45:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960593+0000", 2024-09-16T11:45:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4585000000000001e-05 2024-09-16T11:45:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.256 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89611 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1687552~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.319586776000001, 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031799929999999997, 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89611 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:45:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929211+0000", 2024-09-16T11:45:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929212+0000", 2024-09-16T11:45:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0800000000000001e-07 2024-09-16T11:45:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929213+0000", 2024-09-16T11:45:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6509999999999999e-06 2024-09-16T11:45:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929218+0000", 2024-09-16T11:45:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9899999999999997e-06 2024-09-16T11:45:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929221+0000", 2024-09-16T11:45:14.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1140000000000002e-06 2024-09-16T11:45:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960926+0000", 2024-09-16T11:45:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031704436000000003 2024-09-16T11:45:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.960966+0000", 2024-09-16T11:45:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0055999999999997e-05 2024-09-16T11:45:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961011+0000", 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5374999999999998e-05 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89612 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1777664~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.319573566999999, 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031893013999999997, 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.261 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89612 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:45:14.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929225+0000", 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6e-07 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929226+0000", 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5099999999999999e-06 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929231+0000", 2024-09-16T11:45:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4889999999999997e-06 2024-09-16T11:45:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929234+0000", 2024-09-16T11:45:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7410000000000001e-06 2024-09-16T11:45:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961033+0000", 2024-09-16T11:45:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031799279 2024-09-16T11:45:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961071+0000", 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7759000000000002e-05 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961118+0000", 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6975999999999999e-05 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.265 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89613 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1867776~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:45:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.319561428, 2024-09-16T11:45:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031996359000000002, 2024-09-16T11:45:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89613 2024-09-16T11:45:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929237+0000", 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8999999999999998e-07 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929243+0000", 2024-09-16T11:45:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8719999999999999e-06 2024-09-16T11:45:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929249+0000", 2024-09-16T11:45:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6229999999999999e-06 2024-09-16T11:45:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929253+0000", 2024-09-16T11:45:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:45:14.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961145+0000", 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031892648000000003 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961188+0000", 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2466999999999998e-05 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961233+0000", 2024-09-16T11:45:14.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5049999999999997e-05 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89615 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 1957888~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.319541557999997, 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032076532999999997, 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.270 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89615 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929257+0000", 2024-09-16T11:45:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:45:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929259+0000", 2024-09-16T11:45:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7269999999999999e-06 2024-09-16T11:45:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929264+0000", 2024-09-16T11:45:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.524e-06 2024-09-16T11:45:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929267+0000", 2024-09-16T11:45:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.994e-06 2024-09-16T11:45:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961252+0000", 2024-09-16T11:45:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031985188999999997 2024-09-16T11:45:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961291+0000", 2024-09-16T11:45:14.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8263000000000003e-05 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961333+0000", 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2491999999999999e-05 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89617 4.3 4:c120f4af:::rbd_data.11c15ec25591.000000000000000c:2a [read 2048000~8192] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.319527925000003, 2024-09-16T11:45:14.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032185244000000002, 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89617 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:45:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929270+0000", 2024-09-16T11:45:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0499999999999999e-07 2024-09-16T11:45:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929272+0000", 2024-09-16T11:45:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.615e-06 2024-09-16T11:45:14.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929279+0000", 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5459999999999997e-06 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929282+0000", 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.472e-06 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961361+0000", 2024-09-16T11:45:14.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032078441999999999 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961401+0000", 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0425e-05 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961455+0000", 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4438999999999998e-05 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89637 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4235264~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.319507668, 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032120463000000002, 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89637 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:45:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929290+0000", 2024-09-16T11:45:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929291+0000", 2024-09-16T11:45:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:45:14.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929292+0000", 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3120000000000001e-06 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929295+0000", 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3349999999999998e-06 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929297+0000", 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0549999999999998e-06 2024-09-16T11:45:14.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961346+0000", 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032048920000000002 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961379+0000", 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2533000000000002e-05 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961411+0000", 2024-09-16T11:45:14.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1921000000000002e-05 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89639 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4325376~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.319498588999998, 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032194749000000002, 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.283 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89639 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929299+0000", 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929300+0000", 2024-09-16T11:45:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:45:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929301+0000", 2024-09-16T11:45:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.018e-06 2024-09-16T11:45:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929304+0000", 2024-09-16T11:45:14.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8890000000000002e-06 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929306+0000", 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.883e-06 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961422+0000", 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032116899999999997 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961460+0000", 2024-09-16T11:45:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7265999999999999e-05 2024-09-16T11:45:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961494+0000", 2024-09-16T11:45:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4499000000000003e-05 2024-09-16T11:45:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.287 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89641 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4415488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:45:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.319490666, 2024-09-16T11:45:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032287719999999999, 2024-09-16T11:45:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89641 2024-09-16T11:45:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:45:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929307+0000", 2024-09-16T11:45:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929308+0000", 2024-09-16T11:45:14.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1500000000000001e-07 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929309+0000", 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.02e-06 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929312+0000", 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3000000000000002e-06 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929314+0000", 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0449999999999999e-06 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961521+0000", 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032206835000000003 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961561+0000", 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0580000000000001e-05 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961595+0000", 2024-09-16T11:45:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3725e-05 2024-09-16T11:45:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89643 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4505600~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:45:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.319482227999998, 2024-09-16T11:45:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032377892999999998, 2024-09-16T11:45:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.292 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89643 2024-09-16T11:45:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:45:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.293 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:45:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929316+0000", 2024-09-16T11:45:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0100000000000001e-07 2024-09-16T11:45:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929317+0000", 2024-09-16T11:45:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.161e-06 2024-09-16T11:45:14.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929320+0000", 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1580000000000001e-06 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929322+0000", 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9929999999999998e-06 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961619+0000", 2024-09-16T11:45:14.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032296392 2024-09-16T11:45:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961659+0000", 2024-09-16T11:45:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9805000000000003e-05 2024-09-16T11:45:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961694+0000", 2024-09-16T11:45:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5182999999999998e-05 2024-09-16T11:45:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.296 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89644 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4243456~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:45:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.319473848999998, 2024-09-16T11:45:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032467912000000002, 2024-09-16T11:45:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89644 2024-09-16T11:45:14.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.298 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929324+0000", 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0100000000000001e-07 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929325+0000", 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9899999999999999e-07 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929328+0000", 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7539999999999998e-06 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929330+0000", 2024-09-16T11:45:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3690000000000001e-06 2024-09-16T11:45:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961725+0000", 2024-09-16T11:45:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032394088000000001 2024-09-16T11:45:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961759+0000", 2024-09-16T11:45:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4316999999999997e-05 2024-09-16T11:45:14.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961792+0000", 2024-09-16T11:45:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3284000000000002e-05 2024-09-16T11:45:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89645 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4595712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:45:14.301 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.319465776999998, 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032554313000000001, 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89645 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929332+0000", 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.99e-07 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929333+0000", 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9299999999999996e-07 2024-09-16T11:45:14.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929336+0000", 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8459999999999999e-06 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929340+0000", 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6370000000000001e-06 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961814+0000", 2024-09-16T11:45:14.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032474461000000003 2024-09-16T11:45:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961852+0000", 2024-09-16T11:45:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7877999999999999e-05 2024-09-16T11:45:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961887+0000", 2024-09-16T11:45:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4399e-05 2024-09-16T11:45:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.305 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89646 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4333568~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.31945614, 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032641146000000003, 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89646 2024-09-16T11:45:14.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929342+0000", 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.04e-07 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929343+0000", 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7000000000000003e-07 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929346+0000", 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7350000000000001e-06 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929349+0000", 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7030000000000002e-06 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961916+0000", 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032567634999999998 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961950+0000", 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3510000000000003e-05 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.961983+0000", 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3389000000000002e-05 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89647 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4685824~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.319447738000001, 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032727342, 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.310 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89647 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.929350+0000", 2024-09-16T11:45:14.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931111+0000", 2024-09-16T11:45:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001760462 2024-09-16T11:45:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931114+0000", 2024-09-16T11:45:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.67e-06 2024-09-16T11:45:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931122+0000", 2024-09-16T11:45:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6059999999999997e-06 2024-09-16T11:45:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931126+0000", 2024-09-16T11:45:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-06 2024-09-16T11:45:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962005+0000", 2024-09-16T11:45:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.030879125 2024-09-16T11:45:14.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962044+0000", 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8779000000000003e-05 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962078+0000", 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4010000000000001e-05 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89656 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4784128~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.317597689000003, 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031742976999999999, 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89656 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:45:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931200+0000", 2024-09-16T11:45:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931201+0000", 2024-09-16T11:45:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0699999999999999e-07 2024-09-16T11:45:14.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931202+0000", 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5900000000000005e-07 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931204+0000", 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8439999999999998e-06 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931206+0000", 2024-09-16T11:45:14.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.871e-06 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962877+0000", 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031670831000000003 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962910+0000", 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3405999999999998e-05 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962943+0000", 2024-09-16T11:45:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2858999999999998e-05 2024-09-16T11:45:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89657 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5136384~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:45:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.317589878, 2024-09-16T11:45:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031827938, 2024-09-16T11:45:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.319 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89657 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:45:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931208+0000", 2024-09-16T11:45:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.28e-07 2024-09-16T11:45:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931209+0000", 2024-09-16T11:45:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9099999999999991e-07 2024-09-16T11:45:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931213+0000", 2024-09-16T11:45:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7639999999999999e-06 2024-09-16T11:45:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931217+0000", 2024-09-16T11:45:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4759999999999998e-06 2024-09-16T11:45:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.962964+0000", 2024-09-16T11:45:14.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031747598000000002 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963003+0000", 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8287000000000001e-05 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963036+0000", 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3593999999999997e-05 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89658 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4874240~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:45:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.317562383000002, 2024-09-16T11:45:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031905067000000002, 2024-09-16T11:45:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89658 2024-09-16T11:45:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.324 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:45:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:45:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931236+0000", 2024-09-16T11:45:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4700000000000001e-07 2024-09-16T11:45:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931238+0000", 2024-09-16T11:45:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:45:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931245+0000", 2024-09-16T11:45:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.64e-06 2024-09-16T11:45:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931248+0000", 2024-09-16T11:45:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5719999999999999e-06 2024-09-16T11:45:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963066+0000", 2024-09-16T11:45:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031817326999999999 2024-09-16T11:45:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963107+0000", 2024-09-16T11:45:14.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1220000000000002e-05 2024-09-16T11:45:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963141+0000", 2024-09-16T11:45:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3896999999999997e-05 2024-09-16T11:45:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89659 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 4964352~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:45:14.328 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.317546081000003, 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031984460999999999, 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89659 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:45:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931252+0000", 2024-09-16T11:45:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:45:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931254+0000", 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.776e-06 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931262+0000", 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1680000000000004e-06 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931266+0000", 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.732e-06 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963165+0000", 2024-09-16T11:45:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031898693999999998 2024-09-16T11:45:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963202+0000", 2024-09-16T11:45:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7517e-05 2024-09-16T11:45:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963236+0000", 2024-09-16T11:45:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4230000000000003e-05 2024-09-16T11:45:14.332 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89660 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5226496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:45:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.317528060999997, 2024-09-16T11:45:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032062368000000001, 2024-09-16T11:45:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89660 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931270+0000", 2024-09-16T11:45:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.39e-07 2024-09-16T11:45:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931272+0000", 2024-09-16T11:45:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:45:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931279+0000", 2024-09-16T11:45:14.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:45:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931282+0000", 2024-09-16T11:45:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.721e-06 2024-09-16T11:45:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963266+0000", 2024-09-16T11:45:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.031983971999999999 2024-09-16T11:45:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963300+0000", 2024-09-16T11:45:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3318000000000002e-05 2024-09-16T11:45:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963332+0000", 2024-09-16T11:45:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2768000000000002e-05 2024-09-16T11:45:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.337 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:89661 4.b 4:d0323383:::rbd_data.11c15ec25591.0000000000000009:2a [read 5054464~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e83)", 2024-09-16T11:45:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:45:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 37.317511637000003, 2024-09-16T11:45:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.032139394000000002, 2024-09-16T11:45:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89661 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.931286+0000", 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933104+0000", 2024-09-16T11:45:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00181734 2024-09-16T11:45:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933107+0000", 2024-09-16T11:45:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5140000000000002e-06 2024-09-16T11:45:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933115+0000", 2024-09-16T11:45:14.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.933124+0000", 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8910000000000001e-06 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963354+0000", 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.030230825999999999 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963392+0000", 2024-09-16T11:45:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7765000000000002e-05 2024-09-16T11:45:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:36.963426+0000", 2024-09-16T11:45:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3603000000000003e-05 2024-09-16T11:45:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.344 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.344 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:14.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:45:14.346 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:14.343+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:14.447 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:14.445+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:14.485 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:45:14.485 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:14.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:45:14.547 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:14.546+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:14.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:45:14.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-16T11:45:14.576+0000 7f32e5615200 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-16T11:45:14.648 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:14.646+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:14.649 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:14.649 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:14.649 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:45:14.649 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:45:14.649 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:14.659 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:45:14.749 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:14.747+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:14.768 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:45:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:45:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:45:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19364 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1171456~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:45:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.251991887000003, 2024-09-16T11:45:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049001805000000002, 2024-09-16T11:45:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.769 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19364 2024-09-16T11:45:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:45:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.770 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513278+0000", 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513280+0000", 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7090000000000001e-06 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513286+0000", 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5840000000000002e-06 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513290+0000", 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3680000000000004e-06 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562121+0000", 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048831324000000002 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562195+0000", 2024-09-16T11:45:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3415000000000001e-05 2024-09-16T11:45:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562280+0000", 2024-09-16T11:45:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4975999999999994e-05 2024-09-16T11:45:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.774 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19365 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1490944~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.774 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:45:14.774 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.251972147999993, 2024-09-16T11:45:14.774 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049335349000000001, 2024-09-16T11:45:14.774 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.774 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.774 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.774 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.774 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.774 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19365 2024-09-16T11:45:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:45:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:45:14.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513298+0000", 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513300+0000", 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3020000000000002e-06 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513306+0000", 2024-09-16T11:45:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8780000000000003e-06 2024-09-16T11:45:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513310+0000", 2024-09-16T11:45:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1309999999999999e-06 2024-09-16T11:45:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562322+0000", 2024-09-16T11:45:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049011635999999997 2024-09-16T11:45:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562544+0000", 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022168000000000001 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562633+0000", 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9350000000000003e-05 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.778 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19366 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1916928~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.251955215999999, 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049865073000000003, 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19366 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.779 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:45:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.513315+0000", 2024-09-16T11:45:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515102+0000", 2024-09-16T11:45:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017877419999999999 2024-09-16T11:45:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515106+0000", 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4450000000000001e-06 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515113+0000", 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7680000000000008e-06 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515117+0000", 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:45:14.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.562756+0000", 2024-09-16T11:45:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047638579 2024-09-16T11:45:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563093+0000", 2024-09-16T11:45:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000337516 2024-09-16T11:45:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563180+0000", 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6420000000000003e-05 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19371 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3194880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.250058009, 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048948356999999998, 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.783 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19371 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515212+0000", 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515214+0000", 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5969999999999999e-06 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515219+0000", 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4559999999999999e-06 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515223+0000", 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.563991+0000", 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048768911999999998 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564063+0000", 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2010999999999999e-05 2024-09-16T11:45:14.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564160+0000", 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6617e-05 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19372 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3620864~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.250043792, 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049130502, 2024-09-16T11:45:14.787 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19372 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.788 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:45:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515226+0000", 2024-09-16T11:45:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:45:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515228+0000", 2024-09-16T11:45:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T11:45:14.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515233+0000", 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515237+0000", 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.27e-06 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564199+0000", 2024-09-16T11:45:14.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048961787999999999 2024-09-16T11:45:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564275+0000", 2024-09-16T11:45:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6001000000000006e-05 2024-09-16T11:45:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564356+0000", 2024-09-16T11:45:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1941999999999996e-05 2024-09-16T11:45:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.791 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19373 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3940352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:45:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.250029392000002, 2024-09-16T11:45:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049289588000000002, 2024-09-16T11:45:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.792 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19373 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515240+0000", 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515241+0000", 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515242+0000", 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.528e-06 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515248+0000", 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5110000000000003e-06 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515251+0000", 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2480000000000001e-06 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564388+0000", 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049137313000000002 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564455+0000", 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7049000000000001e-05 2024-09-16T11:45:14.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564530+0000", 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4626e-05 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19374 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 4046848~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.250014988999993, 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049457133, 2024-09-16T11:45:14.796 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19374 2024-09-16T11:45:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:45:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515255+0000", 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515257+0000", 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5740000000000001e-06 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515262+0000", 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2399999999999998e-06 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515265+0000", 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3909999999999998e-06 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564555+0000", 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049290002999999999 2024-09-16T11:45:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564628+0000", 2024-09-16T11:45:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2559999999999996e-05 2024-09-16T11:45:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564712+0000", 2024-09-16T11:45:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4040000000000002e-05 2024-09-16T11:45:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19375 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3301376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:45:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.250000846000006, 2024-09-16T11:45:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049606631999999998, 2024-09-16T11:45:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19375 2024-09-16T11:45:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.801 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515269+0000", 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1399999999999998e-07 2024-09-16T11:45:14.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515271+0000", 2024-09-16T11:45:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5230000000000001e-06 2024-09-16T11:45:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515276+0000", 2024-09-16T11:45:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4650000000000002e-06 2024-09-16T11:45:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515279+0000", 2024-09-16T11:45:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0630000000000002e-06 2024-09-16T11:45:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564738+0000", 2024-09-16T11:45:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049459001000000002 2024-09-16T11:45:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564780+0000", 2024-09-16T11:45:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1693999999999998e-05 2024-09-16T11:45:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564876+0000", 2024-09-16T11:45:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5571999999999999e-05 2024-09-16T11:45:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19376 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 3727360~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:45:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.249986934000006, 2024-09-16T11:45:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049720590000000002, 2024-09-16T11:45:14.805 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19376 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.515283+0000", 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517103+0000", 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001820276 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517108+0000", 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5410000000000002e-06 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517120+0000", 2024-09-16T11:45:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2092e-05 2024-09-16T11:45:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517125+0000", 2024-09-16T11:45:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6829999999999996e-06 2024-09-16T11:45:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564870+0000", 2024-09-16T11:45:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047745040000000002 2024-09-16T11:45:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.564921+0000", 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0538999999999998e-05 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.565003+0000", 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2418999999999998e-05 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.809 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19493 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 860160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.248036407000001, 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049086604999999998, 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19493 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:45:14.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517233+0000", 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517234+0000", 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.37e-07 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517236+0000", 2024-09-16T11:45:14.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7150000000000001e-06 2024-09-16T11:45:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517243+0000", 2024-09-16T11:45:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9489999999999999e-06 2024-09-16T11:45:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517246+0000", 2024-09-16T11:45:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.636e-06 2024-09-16T11:45:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566163+0000", 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048917284999999998 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566237+0000", 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3196999999999997e-05 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566320+0000", 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3286000000000002e-05 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.813 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19494 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1179648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.248015791, 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049259562999999999, 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19494 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.814 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517254+0000", 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:45:14.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517256+0000", 2024-09-16T11:45:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.888e-06 2024-09-16T11:45:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517265+0000", 2024-09-16T11:45:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8629999999999997e-06 2024-09-16T11:45:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517269+0000", 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2039999999999999e-06 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566352+0000", 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049083107000000001 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566429+0000", 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7025000000000002e-05 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566514+0000", 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4121000000000003e-05 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19495 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1286144~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.247996368000003, 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049492880000000003, 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19495 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517273+0000", 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517274+0000", 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517276+0000", 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8950000000000001e-06 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517281+0000", 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3879999999999999e-06 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517285+0000", 2024-09-16T11:45:14.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5930000000000002e-06 2024-09-16T11:45:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566579+0000", 2024-09-16T11:45:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049294175000000003 2024-09-16T11:45:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566670+0000", 2024-09-16T11:45:14.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1096999999999999e-05 2024-09-16T11:45:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566766+0000", 2024-09-16T11:45:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6355999999999997e-05 2024-09-16T11:45:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19496 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1605632~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.825 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:45:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.247981283000001, 2024-09-16T11:45:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049632981999999999, 2024-09-16T11:45:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19496 2024-09-16T11:45:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517288+0000", 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517289+0000", 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517290+0000", 2024-09-16T11:45:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.581e-06 2024-09-16T11:45:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517300+0000", 2024-09-16T11:45:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6660000000000003e-06 2024-09-16T11:45:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517304+0000", 2024-09-16T11:45:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:45:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566789+0000", 2024-09-16T11:45:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049485102000000003 2024-09-16T11:45:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566843+0000", 2024-09-16T11:45:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3937000000000002e-05 2024-09-16T11:45:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566921+0000", 2024-09-16T11:45:14.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8397000000000002e-05 2024-09-16T11:45:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19497 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 114688~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:45:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.247961587999995, 2024-09-16T11:45:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049761367000000001, 2024-09-16T11:45:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.830 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19497 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.517308+0000", 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519116+0000", 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018079439999999999 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519120+0000", 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2760000000000002e-06 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519134+0000", 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3468999999999999e-05 2024-09-16T11:45:14.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519140+0000", 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.516e-06 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566943+0000", 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047802561 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.566982+0000", 2024-09-16T11:45:14.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9269e-05 2024-09-16T11:45:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.567070+0000", 2024-09-16T11:45:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7331999999999997e-05 2024-09-16T11:45:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19504 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 966656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:45:14.834 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.246033100999995, 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049045863000000002, 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19504 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519237+0000", 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519239+0000", 2024-09-16T11:45:14.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.962e-06 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519244+0000", 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5480000000000001e-06 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519248+0000", 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.641e-06 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568150+0000", 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048901864000000003 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568208+0000", 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8399000000000001e-05 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568283+0000", 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4105999999999998e-05 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19505 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1392640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:45:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.246017796000004, 2024-09-16T11:45:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049202304000000002, 2024-09-16T11:45:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19505 2024-09-16T11:45:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.839 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519252+0000", 2024-09-16T11:45:14.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:45:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519254+0000", 2024-09-16T11:45:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.761e-06 2024-09-16T11:45:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519257+0000", 2024-09-16T11:45:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.343e-06 2024-09-16T11:45:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519259+0000", 2024-09-16T11:45:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0559999999999999e-06 2024-09-16T11:45:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568307+0000", 2024-09-16T11:45:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049047445000000002 2024-09-16T11:45:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568382+0000", 2024-09-16T11:45:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5574999999999994e-05 2024-09-16T11:45:14.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568454+0000", 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1799999999999997e-05 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19506 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1712128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.246007758000005, 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.049341532, 2024-09-16T11:45:14.843 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19506 2024-09-16T11:45:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:45:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.519262+0000", 2024-09-16T11:45:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521102+0000", 2024-09-16T11:45:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018404700000000001 2024-09-16T11:45:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521106+0000", 2024-09-16T11:45:14.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8820000000000003e-06 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521119+0000", 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2546e-05 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521125+0000", 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0249999999999999e-06 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568484+0000", 2024-09-16T11:45:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.047359120999999997 2024-09-16T11:45:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568542+0000", 2024-09-16T11:45:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8360999999999999e-05 2024-09-16T11:45:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.568604+0000", 2024-09-16T11:45:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1126999999999999e-05 2024-09-16T11:45:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.847 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:19515 4.12 4:49ac94c1:::rbd_data.11c1a7dbfac1.0000000000000007:f [read 1499136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:45:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.244015473000005, 2024-09-16T11:45:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.048949431000000002, 2024-09-16T11:45:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.848 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 19515 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.521254+0000", 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523130+0000", 2024-09-16T11:45:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00187604 2024-09-16T11:45:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523134+0000", 2024-09-16T11:45:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:45:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523144+0000", 2024-09-16T11:45:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2010000000000005e-06 2024-09-16T11:45:14.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.523149+0000", 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0039999999999999e-06 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570113+0000", 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.046964517999999997 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570157+0000", 2024-09-16T11:45:14.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4056e-05 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.570204+0000", 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6591999999999997e-05 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:44111 4.8 4:17f3f47e:::rbd_data.11c1e170f5f0.000000000000002f:head [delete] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 64.782035191999995, 2024-09-16T11:45:14.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93903426400000001, 2024-09-16T11:45:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 44111 2024-09-16T11:45:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:45:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983235+0000", 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7199999999999998e-07 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983239+0000", 2024-09-16T11:45:14.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9879999999999998e-06 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983248+0000", 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.371e-06 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983253+0000", 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1150000000000002e-06 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983303+0000", 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0380000000000002e-05 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:09.983311+0000", 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7519999999999996e-06 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920618+0000", 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93730672000000004 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920730+0000", 2024-09-16T11:45:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011265900000000001 2024-09-16T11:45:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 7", 2024-09-16T11:45:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.920842+0000", 2024-09-16T11:45:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111638 2024-09-16T11:45:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T11:45:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.921503+0000", 2024-09-16T11:45:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066074299999999999 2024-09-16T11:45:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922208+0000", 2024-09-16T11:45:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00070492899999999997 2024-09-16T11:45:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T11:45:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922233+0000", 2024-09-16T11:45:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5922999999999999e-05 2024-09-16T11:45:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:44:10.922269+0000", 2024-09-16T11:45:14.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5373999999999997e-05 2024-09-16T11:45:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:94478 4.10 4:0e606136:::rbd_header.11c1ac01afc0:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-16T11:45:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:45:05.528279+0000", 2024-09-16T11:45:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 9.2369909349999997, 2024-09-16T11:45:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.92682059800000005, 2024-09-16T11:45:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 94478 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:45:05.528279+0000", 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:45:05.528279+0000", 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:14.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:45:05.528280+0000", 2024-09-16T11:45:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.513e-06 2024-09-16T11:45:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:45:05.528288+0000", 2024-09-16T11:45:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5129999999999999e-06 2024-09-16T11:45:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:14.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:45:05.528305+0000", 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6949999999999999e-05 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:45:05.528315+0000", 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0455000000000001e-05 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:45:05.528379+0000", 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3368999999999996e-05 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:45:05.528387+0000", 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9400000000000002e-06 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:45:05.530565+0000", 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021781109999999999 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:45:05.530573+0000", 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0970000000000006e-06 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:45:06.454631+0000", 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.92405789699999996 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:45:06.454778+0000", 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014768100000000001 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:45:06.455099+0000", 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00032107199999999999 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.864 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.865 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.865 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:14.865 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:14.865 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:14.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:45:14.867 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:14.864+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:14.967 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:14.965+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:15.005 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:15.005 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:45:15.005 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:15.013 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:45:15.098 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:15.096+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:15.153 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:15.153 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:15.153 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:45:15.153 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:45:15.154 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:15.161 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:45:15.169 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:15.167+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:15.270 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:15.268+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:15.300 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:45:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:45:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:45:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16853 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3375104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:45:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.950552259000006, 2024-09-16T11:45:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027791907000000001, 2024-09-16T11:45:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16853 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347568+0000", 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347570+0000", 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347588+0000", 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7914000000000002e-05 2024-09-16T11:45:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347592+0000", 2024-09-16T11:45:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3150000000000002e-06 2024-09-16T11:45:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375199+0000", 2024-09-16T11:45:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027606065999999999 2024-09-16T11:45:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375276+0000", 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7527000000000005e-05 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375359+0000", 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3238000000000005e-05 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16854 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.950522562000003, 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027945148999999999, 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16854 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:45:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347598+0000", 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347600+0000", 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:45:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347606+0000", 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0580000000000003e-06 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347609+0000", 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375386+0000", 2024-09-16T11:45:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027776809 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375461+0000", 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4868000000000002e-05 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375542+0000", 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1253000000000003e-05 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16855 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 4120576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.950506223000005, 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028111094999999999, 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16855 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347614+0000", 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347616+0000", 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347623+0000", 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0099999999999998e-06 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347626+0000", 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2140000000000001e-06 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375569+0000", 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027942649999999999 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375643+0000", 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3949999999999995e-05 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375725+0000", 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1992999999999999e-05 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16856 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3801088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:45:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.950484152000001, 2024-09-16T11:45:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028268272000000001, 2024-09-16T11:45:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16856 2024-09-16T11:45:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347636+0000", 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:45:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347638+0000", 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.86e-06 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347644+0000", 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1739999999999997e-06 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347656+0000", 2024-09-16T11:45:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2198000000000001e-05 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375751+0000", 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028094507000000001 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375824+0000", 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3178999999999998e-05 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375904+0000", 2024-09-16T11:45:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9985000000000001e-05 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16857 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.950457650000004, 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028275403000000001, 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16857 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.46e-07 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347666+0000", 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8739999999999997e-06 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347672+0000", 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1059999999999996e-06 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347676+0000", 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375930+0000", 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028254042 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375937+0000", 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5519999999999999e-06 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16858 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.950440005000004, 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028284592000000001, 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16858 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:45:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347683+0000", 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6809999999999998e-06 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347689+0000", 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347695+0000", 2024-09-16T11:45:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0020000000000003e-06 2024-09-16T11:45:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375958+0000", 2024-09-16T11:45:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028263870999999999 2024-09-16T11:45:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375964+0000", 2024-09-16T11:45:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8209999999999998e-06 2024-09-16T11:45:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16992 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:45:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.948865316999999, 2024-09-16T11:45:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027704776, 2024-09-16T11:45:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16992 2024-09-16T11:45:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349255+0000", 2024-09-16T11:45:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:45:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349260+0000", 2024-09-16T11:45:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5589999999999996e-06 2024-09-16T11:45:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349267+0000", 2024-09-16T11:45:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4019999999999999e-06 2024-09-16T11:45:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349270+0000", 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6780000000000002e-06 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376953+0000", 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682933 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376959+0000", 2024-09-16T11:45:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16993 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.948843991999993, 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027707908999999999, 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16993 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:45:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349278+0000", 2024-09-16T11:45:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.629e-06 2024-09-16T11:45:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349291+0000", 2024-09-16T11:45:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614e-05 2024-09-16T11:45:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349295+0000", 2024-09-16T11:45:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1189999999999999e-06 2024-09-16T11:45:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376978+0000", 2024-09-16T11:45:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682445 2024-09-16T11:45:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376984+0000", 2024-09-16T11:45:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7590000000000003e-06 2024-09-16T11:45:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16994 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.948819981, 2024-09-16T11:45:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027708753999999999, 2024-09-16T11:45:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16994 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349302+0000", 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.722e-06 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349308+0000", 2024-09-16T11:45:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:45:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349312+0000", 2024-09-16T11:45:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5310000000000002e-06 2024-09-16T11:45:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377003+0000", 2024-09-16T11:45:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027691308000000001 2024-09-16T11:45:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377008+0000", 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4550000000000003e-06 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16999 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.948677102999994, 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027705270000000001, 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16999 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.46e-07 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349445+0000", 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.637e-06 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349451+0000", 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9959999999999999e-06 2024-09-16T11:45:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349454+0000", 2024-09-16T11:45:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.647e-06 2024-09-16T11:45:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377142+0000", 2024-09-16T11:45:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027687942 2024-09-16T11:45:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377148+0000", 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7019999999999998e-06 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17000 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.948661595999994, 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.02771881, 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17000 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0600000000000001e-07 2024-09-16T11:45:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349460+0000", 2024-09-16T11:45:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.736e-06 2024-09-16T11:45:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349466+0000", 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2419999999999997e-06 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349470+0000", 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5480000000000002e-06 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377171+0000", 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027701282000000001 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377177+0000", 2024-09-16T11:45:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5960000000000003e-06 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17001 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.948645846000005, 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027739243, 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17001 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:45:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349476+0000", 2024-09-16T11:45:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T11:45:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349482+0000", 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2389999999999999e-06 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349486+0000", 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5980000000000001e-06 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377206+0000", 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027720037999999999 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377213+0000", 2024-09-16T11:45:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3440000000000003e-06 2024-09-16T11:45:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17433 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 5898240~4888] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:45:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.912682192000005, 2024-09-16T11:45:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027709398, 2024-09-16T11:45:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17433 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:45:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385438+0000", 2024-09-16T11:45:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:45:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385439+0000", 2024-09-16T11:45:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5120000000000001e-06 2024-09-16T11:45:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385445+0000", 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4e-06 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385448+0000", 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1669999999999999e-06 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412916+0000", 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027468137 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412988+0000", 2024-09-16T11:45:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1606000000000005e-05 2024-09-16T11:45:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413147+0000", 2024-09-16T11:45:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015924299999999999 2024-09-16T11:45:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17434 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6004736~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.912667601999999, 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027777461, 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17434 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385454+0000", 2024-09-16T11:45:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6729999999999999e-06 2024-09-16T11:45:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385459+0000", 2024-09-16T11:45:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4079999999999997e-06 2024-09-16T11:45:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385463+0000", 2024-09-16T11:45:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.269e-06 2024-09-16T11:45:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413157+0000", 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027694611000000001 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413193+0000", 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5673999999999998e-05 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413230+0000", 2024-09-16T11:45:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6454e-05 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17435 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6324224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.912653449000004, 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929059999999999, 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17435 2024-09-16T11:45:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:45:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:45:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385467+0000", 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385468+0000", 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6300000000000001e-06 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385474+0000", 2024-09-16T11:45:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1299999999999998e-06 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385478+0000", 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2880000000000001e-06 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413257+0000", 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027779608000000001 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413331+0000", 2024-09-16T11:45:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3639000000000004e-05 2024-09-16T11:45:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413395+0000", 2024-09-16T11:45:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4424999999999994e-05 2024-09-16T11:45:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17436 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7176192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.912637786999994, 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028016718999999999, 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17436 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:45:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385484+0000", 2024-09-16T11:45:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5179999999999999e-06 2024-09-16T11:45:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385489+0000", 2024-09-16T11:45:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7049999999999996e-06 2024-09-16T11:45:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385493+0000", 2024-09-16T11:45:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413422+0000", 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929611999999999 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413462+0000", 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9585000000000001e-05 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413499+0000", 2024-09-16T11:45:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6644999999999999e-05 2024-09-16T11:45:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17437 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6750208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:45:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.912623371999999, 2024-09-16T11:45:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028167168999999999, 2024-09-16T11:45:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17437 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:45:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385497+0000", 2024-09-16T11:45:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:45:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385498+0000", 2024-09-16T11:45:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.592e-06 2024-09-16T11:45:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385504+0000", 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4090000000000002e-06 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385513+0000", 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4460000000000007e-06 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413526+0000", 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028012866000000001 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413601+0000", 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5388000000000004e-05 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413663+0000", 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.211e-05 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17438 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7495680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.912600612999995, 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028224439, 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17438 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4900000000000001e-07 2024-09-16T11:45:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385521+0000", 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385526+0000", 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2680000000000002e-06 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385529+0000", 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2040000000000002e-06 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413674+0000", 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028144226000000001 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413710+0000", 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6665000000000002e-05 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413743+0000", 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3155999999999999e-05 2024-09-16T11:45:15.376 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17439 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7921664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.912539459000001, 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028326131000000001, 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17439 2024-09-16T11:45:15.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385581+0000", 2024-09-16T11:45:15.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9299999999999998e-07 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385583+0000", 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6220000000000001e-06 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385592+0000", 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5220000000000007e-06 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385595+0000", 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8669999999999998e-06 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413771+0000", 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028175875999999999 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413845+0000", 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4382999999999996e-05 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413906+0000", 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1367999999999999e-05 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17440 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 79.912522250999999, 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028391117, 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:15.381 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17440 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:45:15.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385598+0000", 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385603+0000", 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:15.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385607+0000", 2024-09-16T11:45:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4869999999999998e-06 2024-09-16T11:45:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385609+0000", 2024-09-16T11:45:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2759999999999999e-06 2024-09-16T11:45:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413916+0000", 2024-09-16T11:45:15.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028307253000000001 2024-09-16T11:45:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413955+0000", 2024-09-16T11:45:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8652000000000002e-05 2024-09-16T11:45:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413989+0000", 2024-09-16T11:45:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3587999999999998e-05 2024-09-16T11:45:15.385 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.386 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.386 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.386 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:15.386 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:15.386 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:15.387 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:45:15.388 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:15.385+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:15.488 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:15.486+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:15.498 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:15.498 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:15.507 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:45:15.546 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:45:15.574 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T11:45:15.589 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:15.587+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:15.609 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:15.609 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:15.610 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:15.617 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:45:15.678 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:45:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:45:15.689 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:15.687+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:15.693 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:45:15.694 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:45:15.710 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:45:15.710 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:45:15.710 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T11:45:15.721 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:15.721 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:15.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:15.722 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:15.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.722 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.722 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:15.722 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357731666999996, 2024-09-16T11:45:15.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:45:15.722 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.722 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.722 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.722 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.722 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.722 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:15.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:45:15.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:45:15.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:45:15.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:45:15.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:45:15.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:45:15.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:45:15.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:45:15.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:45:15.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.726 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:15.727 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357709638000003, 2024-09-16T11:45:15.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:45:15.727 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.727 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.727 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:45:15.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.727 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:15.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:15.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:15.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:45:15.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:45:15.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:45:15.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:45:15.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:45:15.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:45:15.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357694873, 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.731 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:15.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:15.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:45:15.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:45:15.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:45:15.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:45:15.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:45:15.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:45:15.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:45:15.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:45:15.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357674805000002, 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357643968999994, 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:15.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:15.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:15.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:45:15.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:45:15.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:45:15.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:45:15.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:45:15.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:45:15.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:45:15.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:45:15.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:45:15.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:45:15.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:45:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:45:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:45:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357624448999999, 2024-09-16T11:45:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:45:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:45:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:45:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:45:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:45:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:45:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:45:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:45:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357596869999995, 2024-09-16T11:45:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:45:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:45:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:45:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:45:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357578175, 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:45:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:45:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:45:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:45:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:45:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:45:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:45:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:45:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:45:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:45:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:45:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:45:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357563166000006, 2024-09-16T11:45:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:45:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:45:15.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.759 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:15.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:15.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:45:15.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:45:15.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:45:15.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:45:15.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:45:15.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:45:15.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:45:15.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:45:15.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:45:15.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:45:15.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:45:15.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:45:15.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:45:15.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:45:15.762 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357531068, 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:45:15.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.764 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:15.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:15.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:45:15.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:45:15.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:45:15.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:45:15.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:45:15.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:45:15.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:45:15.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:45:15.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:45:15.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.768 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.768 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:15.768 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357507448999996, 2024-09-16T11:45:15.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:45:15.768 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.768 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.768 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:45:15.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.768 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:15.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:15.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:45:15.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:45:15.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:45:15.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:45:15.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:45:15.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:45:15.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:45:15.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:45:15.772 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.772 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.772 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.772 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.772 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:15.772 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357492719999996, 2024-09-16T11:45:15.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:45:15.773 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.773 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.773 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.773 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.773 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.773 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:45:15.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.773 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:15.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:15.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:15.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:45:15.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:45:15.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:45:15.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:45:15.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:45:15.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:45:15.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:45:15.777 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.777 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.777 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.777 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.777 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:15.777 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357470895999995, 2024-09-16T11:45:15.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:45:15.777 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.777 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:15.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:15.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:45:15.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:45:15.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:45:15.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:45:15.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:45:15.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:45:15.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:45:15.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:45:15.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:45:15.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:45:15.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:45:15.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:45:15.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:45:15.781 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357456245999998, 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.782 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:45:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:45:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:45:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:45:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:45:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:45:15.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:45:15.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:45:15.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:45:15.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:45:15.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357434673, 2024-09-16T11:45:15.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:15.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:15.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:45:15.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:45:15.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:45:15.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:45:15.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:45:15.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:45:15.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:45:15.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.790 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.791 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.791 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.791 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:15.791 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357415227000004, 2024-09-16T11:45:15.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:45:15.791 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.791 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.791 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.791 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.791 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.791 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:45:15.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.792 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:15.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:15.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:45:15.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:45:15.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:45:15.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:45:15.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:45:15.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:15.795 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357378530000005, 2024-09-16T11:45:15.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:45:15.796 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.796 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.796 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:45:15.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.796 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:15.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:15.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:45:15.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:45:15.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:45:15.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:45:15.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:45:15.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:45:15.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:45:15.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357356584000001, 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:45:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:15.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:15.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:45:15.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:45:15.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:45:15.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:45:15.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:45:15.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:45:15.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:45:15.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:45:15.804 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.804 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.804 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357334698000003, 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:45:15.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:45:15.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:45:15.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:45:15.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:45:15.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:45:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:45:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.357319978999996, 2024-09-16T11:45:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:45:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:15.809 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:15.810 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:15.810 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:15.810 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:15.810 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:45:15.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.810 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:15.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:15.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:15.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:15.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:45:15.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:45:15.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:15.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:45:15.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:45:15.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.813 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.814 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.814 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:15.814 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:15.814 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:15.815 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:45:15.815 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:45:15.816 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:45:15.816 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:45:15.816 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:45:15.817 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:45:15.817 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:45:15.817 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:45:15.818 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:45:15.818 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:45:15.818 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:45:15.819 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:15.813+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:15.851 INFO:teuthology.orchestra.run.smithi070.stdout:4.c 2024-09-16T11:45:15.852 INFO:teuthology.orchestra.run.smithi070.stdout:4.f 2024-09-16T11:45:15.852 INFO:teuthology.orchestra.run.smithi070.stdout:4.1d 2024-09-16T11:45:15.852 INFO:teuthology.orchestra.run.smithi070.stdout:4.1b 2024-09-16T11:45:15.852 INFO:teuthology.orchestra.run.smithi070.stdout:4.1a 2024-09-16T11:45:15.852 INFO:teuthology.orchestra.run.smithi070.stdout:4.19 2024-09-16T11:45:15.852 INFO:teuthology.orchestra.run.smithi070.stdout:4.17 2024-09-16T11:45:15.852 INFO:teuthology.orchestra.run.smithi070.stdout:4.16 2024-09-16T11:45:15.852 INFO:teuthology.orchestra.run.smithi070.stdout:4.14 2024-09-16T11:45:15.852 INFO:teuthology.orchestra.run.smithi070.stdout:4.11 2024-09-16T11:45:15.852 INFO:teuthology.orchestra.run.smithi070.stdout:4.10 2024-09-16T11:45:15.852 INFO:teuthology.orchestra.run.smithi070.stdout:4.0 2024-09-16T11:45:15.852 INFO:teuthology.orchestra.run.smithi070.stdout:3.2 2024-09-16T11:45:15.852 INFO:teuthology.orchestra.run.smithi070.stdout:2.1 2024-09-16T11:45:15.916 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:15.914+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:15.917 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:15.917 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:15.925 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:45:16.017 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:16.014+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:16.024 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:16.024 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:16.034 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:45:16.117 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:16.115+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:16.173 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:16.173 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:16.181 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:45:16.218 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:16.216+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:16.279 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:16.279 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:16.287 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:45:16.330 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:16.329+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:16.376 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op export --pgid 4.1b --file /var/log/ceph/exp.4.1b.3 2024-09-16T11:45:16.454 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:16.454 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:16.455 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:45:16.455 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:16.453+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:16.555 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:16.553+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:16.598 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:16.598 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:16.608 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:45:16.656 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:16.655+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:16.715 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:16.715 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:16.724 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:45:16.765 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:16.764+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:16.858 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:16.857+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:16.861 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:16.861 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:16.873 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:45:16.958 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:16.957+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:17.013 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:17.013 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:17.021 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:45:17.062 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:17.061+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:17.120 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:17.120 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:17.128 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:45:17.172 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:17.170+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:17.226 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:17.226 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:17.234 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:45:17.277 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:17.275+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:17.362 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:17.360+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:17.373 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:17.373 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:17.382 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:45:17.464 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:17.461+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:17.478 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:17.478 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:17.487 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:45:17.549 INFO:teuthology.orchestra.run.smithi070.stderr:Exporting 4.1b info 4.1b( v 108'346 (0'0,108'346] local-lis/les=107/108 n=39 ec=28/28 lis/c=107/107 les/c/f=108/108/0 sis=107) 2024-09-16T11:45:17.560 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:d857754c:::rbd_data.11c1b86c78bf.000000000000021d:head# 2024-09-16T11:45:17.560 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:d86120ff:::rbd_data.11c1cbbf29d.0000000000000026:head# 2024-09-16T11:45:17.560 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:d867928f:::rbd_data.11c1b86c78bf.000000000000023f:head# 2024-09-16T11:45:17.561 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:d87206d0:::rbd_data.11c17a9d794a.0000000000000173:head# 2024-09-16T11:45:17.561 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:d89ddbb6:::rbd_data.11c17a9d794a.0000000000000166:head# 2024-09-16T11:45:17.561 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:d929272c:::rbd_data.11c17a9d794a.0000000000000150:head# 2024-09-16T11:45:17.561 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:d982b7e1:::rbd_data.11c17a9d794a.00000000000000c2:head# 2024-09-16T11:45:17.561 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:d9a074e9:::rbd_data.11c17a9d794a.0000000000000153:head# 2024-09-16T11:45:17.561 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:d9ffc0e1:::rbd_data.11c1ac01afc0.000000000000003f:head# 2024-09-16T11:45:17.561 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:da3658b0:::rbd_data.11c1b86c78bf.0000000000000180:head# 2024-09-16T11:45:17.562 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:da4bcbba:::rbd_data.11c130c94979.000000000000000d:6# 2024-09-16T11:45:17.563 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:da4bcbba:::rbd_data.11c130c94979.000000000000000d:head# 2024-09-16T11:45:17.563 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:da8d6b0b:::rbd_data.11c1b86c78bf.000000000000023e:head# 2024-09-16T11:45:17.563 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:da9fad7f:::rbd_data.11c17a9d794a.0000000000000136:head# 2024-09-16T11:45:17.563 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:head# 2024-09-16T11:45:17.564 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:17.563+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:17.570 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:db62f8f9:::rbd_data.11c1b86c78bf.00000000000001b6:head# 2024-09-16T11:45:17.570 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dbc18110:::rbd_data.11c11778572.000000000000000e:head# 2024-09-16T11:45:17.570 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dbead158:::rbd_data.11c17a9d794a.0000000000000109:head# 2024-09-16T11:45:17.570 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dc9de789:::rbd_data.11c17a9d794a.0000000000000122:head# 2024-09-16T11:45:17.570 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dcbdc6eb:::rbd_data.11c17a9d794a.00000000000000f9:head# 2024-09-16T11:45:17.570 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dcc5f862:::rbd_data.11c130c94979.000000000000000b:6# 2024-09-16T11:45:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dcc5f862:::rbd_data.11c130c94979.000000000000000b:head# 2024-09-16T11:45:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dd0bf4c8:::rbd_data.11c17a9d794a.0000000000000172:head# 2024-09-16T11:45:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dd7391d6:::rbd_data.11c17a9d794a.000000000000004c:head# 2024-09-16T11:45:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:ddec7dc0:::rbd_data.11c17a9d794a.000000000000015a:head# 2024-09-16T11:45:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:de2906cb:::rbd_data.11c17a9d794a.00000000000000e3:head# 2024-09-16T11:45:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:de2ddba8:::rbd_data.11c1b86c78bf.000000000000024b:head# 2024-09-16T11:45:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:de51de82:::rbd_data.11c17a9d794a.00000000000000dd:head# 2024-09-16T11:45:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:de84200e:::rbd_data.11c1ac01afc0.0000000000000004:head# 2024-09-16T11:45:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:de86f447:::rbd_data.11c11778572.0000000000000007:head# 2024-09-16T11:45:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:df421a3f:::rbd_header.11c1e24d7fee:head# 2024-09-16T11:45:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:df63fd3b:::rbd_data.11c1b86c78bf.000000000000022d:head# 2024-09-16T11:45:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dfadecd0:::rbd_data.11c1b86c78bf.0000000000000228:head# 2024-09-16T11:45:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dfca7b45:::rbd_data.11c17a9d794a.000000000000017d:head# 2024-09-16T11:45:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5# 2024-09-16T11:45:17.575 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:head# 2024-09-16T11:45:17.575 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dfdc27f6:::rbd_data.11c1b86c78bf.000000000000014e:head# 2024-09-16T11:45:17.575 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dfe5f588:::rbd_data.11c12816a66b.000000000000000d:head# 2024-09-16T11:45:17.577 INFO:teuthology.orchestra.run.smithi070.stderr:Read #4:dff4be41:::rbd_data.11c12e336783.0000000000000002:head# 2024-09-16T11:45:17.578 INFO:teuthology.orchestra.run.smithi070.stderr:Export successful 2024-09-16T11:45:17.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:45:17.593 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:17.594 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:17.603 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:45:17.665 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:17.664+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:17.748 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:17.749 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:17.757 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:45:17.802 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-16T11:45:17.800+0000 7fe0457f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33329) UID: 0 2024-09-16T11:45:17.819 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:45:17.857 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T11:45:17.857 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:45:17.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:45:17.868 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:17.866+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:17.936 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T11:45:17.936 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:45:17.952 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T11:45:17.952 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:45:17.967 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T11:45:17.967 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:45:17.967 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T11:45:17.968 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:17.966+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:17.983 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T11:45:17.983 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:45:17.998 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T11:45:17.998 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:45:18.007 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:18.007 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:18.007 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T11:45:18.007 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T11:45:18.007 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:18.014 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T11:45:18.014 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:45:18.016 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:45:18.030 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T11:45:18.030 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:45:18.033 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:45:18.033 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:45:18.075 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-16T11:45:18.073+0000 7f21671f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33332) UID: 0 2024-09-16T11:45:18.106 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --force --op remove --pgid 4.1b 2024-09-16T11:45:18.130 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:18.131 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T11:45:18.131 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T11:45:18.131 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T11:45:18.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.131 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16853 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3375104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.131 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:45:18.131 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.780668848000005, 2024-09-16T11:45:18.131 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027791907000000001, 2024-09-16T11:45:18.131 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.131 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:18.131 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.131 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.132 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.132 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16853 2024-09-16T11:45:18.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.132 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.132 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.132 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:45:18.132 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.132 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.132 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347567+0000", 2024-09-16T11:45:18.133 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.133 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.133 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347568+0000", 2024-09-16T11:45:18.133 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:45:18.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.133 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.133 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347570+0000", 2024-09-16T11:45:18.133 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2680000000000001e-06 2024-09-16T11:45:18.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347588+0000", 2024-09-16T11:45:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7914000000000002e-05 2024-09-16T11:45:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347592+0000", 2024-09-16T11:45:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3150000000000002e-06 2024-09-16T11:45:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375199+0000", 2024-09-16T11:45:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027606065999999999 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375276+0000", 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7527000000000005e-05 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375359+0000", 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3238000000000005e-05 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.136 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16854 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.136 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:45:18.136 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.780639151000003, 2024-09-16T11:45:18.136 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027945148999999999, 2024-09-16T11:45:18.136 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.136 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:18.136 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.136 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.136 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.136 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16854 2024-09-16T11:45:18.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.136 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:45:18.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347597+0000", 2024-09-16T11:45:18.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347598+0000", 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347600+0000", 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347606+0000", 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0580000000000003e-06 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347609+0000", 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375386+0000", 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027776809 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375461+0000", 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4868000000000002e-05 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375542+0000", 2024-09-16T11:45:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1253000000000003e-05 2024-09-16T11:45:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16855 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 4120576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:45:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.780622812000004, 2024-09-16T11:45:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028111094999999999, 2024-09-16T11:45:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16855 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.141 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347613+0000", 2024-09-16T11:45:18.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347614+0000", 2024-09-16T11:45:18.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:45:18.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347616+0000", 2024-09-16T11:45:18.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:45:18.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347623+0000", 2024-09-16T11:45:18.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0099999999999998e-06 2024-09-16T11:45:18.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347626+0000", 2024-09-16T11:45:18.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2140000000000001e-06 2024-09-16T11:45:18.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375569+0000", 2024-09-16T11:45:18.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027942649999999999 2024-09-16T11:45:18.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:18.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375643+0000", 2024-09-16T11:45:18.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3949999999999995e-05 2024-09-16T11:45:18.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375725+0000", 2024-09-16T11:45:18.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1992999999999999e-05 2024-09-16T11:45:18.144 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.144 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.145 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.145 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16856 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 3801088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.145 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:45:18.145 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.780600741000001, 2024-09-16T11:45:18.145 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028268272000000001, 2024-09-16T11:45:18.145 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.145 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:18.145 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.145 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.145 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.145 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16856 2024-09-16T11:45:18.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.146 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.146 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.146 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:45:18.146 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.146 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.146 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347635+0000", 2024-09-16T11:45:18.146 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.147 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.147 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347636+0000", 2024-09-16T11:45:18.147 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:45:18.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.147 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.147 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347638+0000", 2024-09-16T11:45:18.147 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.86e-06 2024-09-16T11:45:18.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.147 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.147 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347644+0000", 2024-09-16T11:45:18.147 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1739999999999997e-06 2024-09-16T11:45:18.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.148 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.148 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347656+0000", 2024-09-16T11:45:18.148 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2198000000000001e-05 2024-09-16T11:45:18.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.148 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.148 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375751+0000", 2024-09-16T11:45:18.148 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028094507000000001 2024-09-16T11:45:18.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.148 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:18.149 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375824+0000", 2024-09-16T11:45:18.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3178999999999998e-05 2024-09-16T11:45:18.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.149 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.149 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375904+0000", 2024-09-16T11:45:18.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9985000000000001e-05 2024-09-16T11:45:18.149 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.149 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.149 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16857 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.780574239000003, 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028275403000000001, 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16857 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347662+0000", 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.46e-07 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347666+0000", 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8739999999999997e-06 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347672+0000", 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1059999999999996e-06 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347676+0000", 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.153 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375930+0000", 2024-09-16T11:45:18.153 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028254042 2024-09-16T11:45:18.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.153 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.153 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375937+0000", 2024-09-16T11:45:18.153 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5519999999999999e-06 2024-09-16T11:45:18.153 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.153 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.153 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.153 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16858 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.154 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:18.154 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.780556594000004, 2024-09-16T11:45:18.154 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028284592000000001, 2024-09-16T11:45:18.154 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.154 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:18.154 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.154 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.154 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.154 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16858 2024-09-16T11:45:18.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.154 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.154 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347680+0000", 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347683+0000", 2024-09-16T11:45:18.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6809999999999998e-06 2024-09-16T11:45:18.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347689+0000", 2024-09-16T11:45:18.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:45:18.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.347695+0000", 2024-09-16T11:45:18.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0020000000000003e-06 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375958+0000", 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028263870999999999 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.375964+0000", 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8209999999999998e-06 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16992 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:45:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.778981905999999, 2024-09-16T11:45:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027704776, 2024-09-16T11:45:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16992 2024-09-16T11:45:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.159 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:45:18.159 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.159 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349254+0000", 2024-09-16T11:45:18.159 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.159 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349255+0000", 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349260+0000", 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5589999999999996e-06 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349267+0000", 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4019999999999999e-06 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.161 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349270+0000", 2024-09-16T11:45:18.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6780000000000002e-06 2024-09-16T11:45:18.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.161 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.161 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376953+0000", 2024-09-16T11:45:18.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682933 2024-09-16T11:45:18.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.161 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.161 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376959+0000", 2024-09-16T11:45:18.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:45:18.162 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.162 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.162 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.162 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16993 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.162 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:18.162 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.778960581000007, 2024-09-16T11:45:18.162 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027707908999999999, 2024-09-16T11:45:18.162 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.162 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:18.162 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.162 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16993 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.164 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.164 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.164 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349276+0000", 2024-09-16T11:45:18.164 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:45:18.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.164 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.164 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.164 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349278+0000", 2024-09-16T11:45:18.164 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.629e-06 2024-09-16T11:45:18.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.164 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.164 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.164 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349291+0000", 2024-09-16T11:45:18.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3614e-05 2024-09-16T11:45:18.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349295+0000", 2024-09-16T11:45:18.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1189999999999999e-06 2024-09-16T11:45:18.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376978+0000", 2024-09-16T11:45:18.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027682445 2024-09-16T11:45:18.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.376984+0000", 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7590000000000003e-06 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16994 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.778936569999999, 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027708753999999999, 2024-09-16T11:45:18.166 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16994 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.168 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:18.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.168 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.168 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349300+0000", 2024-09-16T11:45:18.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:45:18.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.168 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.168 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349302+0000", 2024-09-16T11:45:18.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.722e-06 2024-09-16T11:45:18.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349308+0000", 2024-09-16T11:45:18.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:45:18.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349312+0000", 2024-09-16T11:45:18.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5310000000000002e-06 2024-09-16T11:45:18.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377003+0000", 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027691308000000001 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377008+0000", 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4550000000000003e-06 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.170 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:16999 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.778793691999994, 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027705270000000001, 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 16999 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349443+0000", 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.46e-07 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349445+0000", 2024-09-16T11:45:18.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.637e-06 2024-09-16T11:45:18.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349451+0000", 2024-09-16T11:45:18.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9959999999999999e-06 2024-09-16T11:45:18.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349454+0000", 2024-09-16T11:45:18.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.647e-06 2024-09-16T11:45:18.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377142+0000", 2024-09-16T11:45:18.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027687942 2024-09-16T11:45:18.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377148+0000", 2024-09-16T11:45:18.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7019999999999998e-06 2024-09-16T11:45:18.174 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.174 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.174 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.175 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17000 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.175 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:18.175 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.778778184999993, 2024-09-16T11:45:18.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.02771881, 2024-09-16T11:45:18.175 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.176 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:18.176 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.176 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.177 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.177 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17000 2024-09-16T11:45:18.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.177 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:18.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.178 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:18.178 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349458+0000", 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0600000000000001e-07 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349460+0000", 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.736e-06 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349466+0000", 2024-09-16T11:45:18.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2419999999999997e-06 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349470+0000", 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5480000000000002e-06 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377171+0000", 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027701282000000001 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377177+0000", 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5960000000000003e-06 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17001 4.1c 4.8c8f681c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.778762435000004, 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027739243, 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.181 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.182 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.182 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17001 2024-09-16T11:45:18.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.182 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.182 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.182 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.182 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:18.182 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.182 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.182 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.182 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:18.182 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349474+0000", 2024-09-16T11:45:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8400000000000005e-07 2024-09-16T11:45:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349476+0000", 2024-09-16T11:45:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T11:45:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349482+0000", 2024-09-16T11:45:18.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2389999999999999e-06 2024-09-16T11:45:18.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.349486+0000", 2024-09-16T11:45:18.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5980000000000001e-06 2024-09-16T11:45:18.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377206+0000", 2024-09-16T11:45:18.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027720037999999999 2024-09-16T11:45:18.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.185 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.185 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.377213+0000", 2024-09-16T11:45:18.185 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3440000000000003e-06 2024-09-16T11:45:18.185 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.185 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.185 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.185 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17433 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 5898240~4888] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.185 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:45:18.185 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.742798781000005, 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027709398, 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17433 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385437+0000", 2024-09-16T11:45:18.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385438+0000", 2024-09-16T11:45:18.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:45:18.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385439+0000", 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5120000000000001e-06 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385445+0000", 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4e-06 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385448+0000", 2024-09-16T11:45:18.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1669999999999999e-06 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412916+0000", 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027468137 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.412988+0000", 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1606000000000005e-05 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413147+0000", 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015924299999999999 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17434 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6004736~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.742784190999998, 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027777461, 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.190 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17434 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.191 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.192 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385452+0000", 2024-09-16T11:45:18.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T11:45:18.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.192 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.192 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385454+0000", 2024-09-16T11:45:18.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6729999999999999e-06 2024-09-16T11:45:18.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.192 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.192 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385459+0000", 2024-09-16T11:45:18.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4079999999999997e-06 2024-09-16T11:45:18.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385463+0000", 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.269e-06 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413157+0000", 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027694611000000001 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413193+0000", 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5673999999999998e-05 2024-09-16T11:45:18.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.194 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.194 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413230+0000", 2024-09-16T11:45:18.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6454e-05 2024-09-16T11:45:18.194 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.194 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.194 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.194 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17435 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6324224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.194 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:45:18.194 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.742770038000003, 2024-09-16T11:45:18.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929059999999999, 2024-09-16T11:45:18.195 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.195 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:18.195 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.195 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.195 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.195 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17435 2024-09-16T11:45:18.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.195 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.195 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.195 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.195 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:45:18.195 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.196 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385466+0000", 2024-09-16T11:45:18.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.196 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385467+0000", 2024-09-16T11:45:18.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:45:18.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385468+0000", 2024-09-16T11:45:18.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6300000000000001e-06 2024-09-16T11:45:18.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385474+0000", 2024-09-16T11:45:18.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1299999999999998e-06 2024-09-16T11:45:18.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385478+0000", 2024-09-16T11:45:18.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2880000000000001e-06 2024-09-16T11:45:18.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413257+0000", 2024-09-16T11:45:18.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027779608000000001 2024-09-16T11:45:18.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:18.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413331+0000", 2024-09-16T11:45:18.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3639000000000004e-05 2024-09-16T11:45:18.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.199 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.199 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413395+0000", 2024-09-16T11:45:18.199 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4424999999999994e-05 2024-09-16T11:45:18.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.199 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.199 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17436 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7176192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.199 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:18.199 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.742754375999993, 2024-09-16T11:45:18.199 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028016718999999999, 2024-09-16T11:45:18.200 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.200 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:18.200 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.200 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.200 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.200 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17436 2024-09-16T11:45:18.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.200 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.200 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.200 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:18.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:18.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385482+0000", 2024-09-16T11:45:18.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:45:18.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385484+0000", 2024-09-16T11:45:18.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5179999999999999e-06 2024-09-16T11:45:18.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385489+0000", 2024-09-16T11:45:18.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7049999999999996e-06 2024-09-16T11:45:18.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385493+0000", 2024-09-16T11:45:18.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T11:45:18.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413422+0000", 2024-09-16T11:45:18.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.027929611999999999 2024-09-16T11:45:18.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:18.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413462+0000", 2024-09-16T11:45:18.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9585000000000001e-05 2024-09-16T11:45:18.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.204 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413499+0000", 2024-09-16T11:45:18.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6644999999999999e-05 2024-09-16T11:45:18.204 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.204 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.204 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.204 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17437 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 6750208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.204 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:45:18.205 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.742739960999998, 2024-09-16T11:45:18.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028167168999999999, 2024-09-16T11:45:18.205 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.205 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:18.205 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.205 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.205 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.205 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17437 2024-09-16T11:45:18.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.205 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385496+0000", 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385497+0000", 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385498+0000", 2024-09-16T11:45:18.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.592e-06 2024-09-16T11:45:18.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385504+0000", 2024-09-16T11:45:18.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4090000000000002e-06 2024-09-16T11:45:18.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385513+0000", 2024-09-16T11:45:18.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4460000000000007e-06 2024-09-16T11:45:18.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413526+0000", 2024-09-16T11:45:18.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028012866000000001 2024-09-16T11:45:18.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:18.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413601+0000", 2024-09-16T11:45:18.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5388000000000004e-05 2024-09-16T11:45:18.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413663+0000", 2024-09-16T11:45:18.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.211e-05 2024-09-16T11:45:18.209 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.209 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.209 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.209 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17438 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7495680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.209 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:18.209 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.742717201999994, 2024-09-16T11:45:18.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028224439, 2024-09-16T11:45:18.210 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.210 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:18.210 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.210 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.210 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.210 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17438 2024-09-16T11:45:18.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.210 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.210 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.210 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:18.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.211 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.211 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:18.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.211 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.211 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385519+0000", 2024-09-16T11:45:18.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4900000000000001e-07 2024-09-16T11:45:18.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.211 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385521+0000", 2024-09-16T11:45:18.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5710000000000001e-06 2024-09-16T11:45:18.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385526+0000", 2024-09-16T11:45:18.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2680000000000002e-06 2024-09-16T11:45:18.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385529+0000", 2024-09-16T11:45:18.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2040000000000002e-06 2024-09-16T11:45:18.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.213 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413674+0000", 2024-09-16T11:45:18.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028144226000000001 2024-09-16T11:45:18.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:18.213 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413710+0000", 2024-09-16T11:45:18.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6665000000000002e-05 2024-09-16T11:45:18.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413743+0000", 2024-09-16T11:45:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3155999999999999e-05 2024-09-16T11:45:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17439 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 7921664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:45:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.742656048000001, 2024-09-16T11:45:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028326131000000001, 2024-09-16T11:45:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17439 2024-09-16T11:45:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:45:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385580+0000", 2024-09-16T11:45:18.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385581+0000", 2024-09-16T11:45:18.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9299999999999998e-07 2024-09-16T11:45:18.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385583+0000", 2024-09-16T11:45:18.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6220000000000001e-06 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385592+0000", 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5220000000000007e-06 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385595+0000", 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8669999999999998e-06 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413771+0000", 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028175875999999999 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413845+0000", 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4382999999999996e-05 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413906+0000", 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1367999999999999e-05 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.218 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.4545.0:17440 4.1c 4:3816f131:::rbd_data.11c1a7dbfac1.0000000000000000:f [read 8347648~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 82.742638839999998, 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028391117, 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.4545", 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 17440 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.219 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T11:45:18.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T11:45:18.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:45:18.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T11:45:18.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385597+0000", 2024-09-16T11:45:18.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T11:45:18.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T11:45:18.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385598+0000", 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385603+0000", 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385607+0000", 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4869999999999998e-06 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.385609+0000", 2024-09-16T11:45:18.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2759999999999999e-06 2024-09-16T11:45:18.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T11:45:18.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413916+0000", 2024-09-16T11:45:18.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.028307253000000001 2024-09-16T11:45:18.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T11:45:18.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413955+0000", 2024-09-16T11:45:18.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8652000000000002e-05 2024-09-16T11:45:18.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T11:45:18.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T11:45:18.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T11:45:18.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T11:43:55.413989+0000", 2024-09-16T11:45:18.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3587999999999998e-05 2024-09-16T11:45:18.223 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.223 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.223 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.223 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T11:45:18.223 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T11:45:18.223 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:18.225 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:45:18.226 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:18.223+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:18.325 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:18.325 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:18.325 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:18.326 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:18.326 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:18.323+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:18.333 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:45:18.426 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:18.425+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:18.431 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:18.432 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:18.440 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:45:18.527 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:18.526+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:18.580 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:18.580 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17725 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216463480000002, 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030287115, 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17725 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361198+0000", 2024-09-16T11:45:18.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361200+0000", 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.928e-06 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361206+0000", 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361212+0000", 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2809999999999997e-06 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391402+0000", 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030189639000000001 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391438+0000", 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5657000000000001e-05 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391485+0000", 2024-09-16T11:45:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7416999999999997e-05 2024-09-16T11:45:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17726 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216441450999994, 2024-09-16T11:45:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030367167, 2024-09-16T11:45:18.585 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17726 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361220+0000", 2024-09-16T11:45:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-16T11:45:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361222+0000", 2024-09-16T11:45:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.671e-06 2024-09-16T11:45:18.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361228+0000", 2024-09-16T11:45:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6359999999999999e-06 2024-09-16T11:45:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361231+0000", 2024-09-16T11:45:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-16T11:45:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391505+0000", 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030273458999999999 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391542+0000", 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7319e-05 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391587+0000", 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5142999999999998e-05 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17727 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2170880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216426686000005, 2024-09-16T11:45:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030456888000000001, 2024-09-16T11:45:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17727 2024-09-16T11:45:18.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361235+0000", 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361241+0000", 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361247+0000", 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9429999999999997e-06 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361251+0000", 2024-09-16T11:45:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:45:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391607+0000", 2024-09-16T11:45:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030356662 2024-09-16T11:45:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391645+0000", 2024-09-16T11:45:18.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7839999999999997e-05 2024-09-16T11:45:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391692+0000", 2024-09-16T11:45:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.638e-05 2024-09-16T11:45:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17728 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2596864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:18.594 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216406617999994, 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030532957999999999, 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17728 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361255+0000", 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361257+0000", 2024-09-16T11:45:18.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7710000000000001e-06 2024-09-16T11:45:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361268+0000", 2024-09-16T11:45:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1214e-05 2024-09-16T11:45:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361278+0000", 2024-09-16T11:45:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4239999999999999e-06 2024-09-16T11:45:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391705+0000", 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030427478000000001 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391746+0000", 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0546000000000001e-05 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391788+0000", 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2154000000000003e-05 2024-09-16T11:45:18.598 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17729 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2916352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216375782, 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030611412000000001, 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.599 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17729 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361286+0000", 2024-09-16T11:45:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:45:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361288+0000", 2024-09-16T11:45:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9549999999999999e-06 2024-09-16T11:45:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361295+0000", 2024-09-16T11:45:18.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2660000000000004e-06 2024-09-16T11:45:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361301+0000", 2024-09-16T11:45:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.643e-06 2024-09-16T11:45:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391816+0000", 2024-09-16T11:45:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030514761000000001 2024-09-16T11:45:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391852+0000", 2024-09-16T11:45:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6157e-05 2024-09-16T11:45:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391897+0000", 2024-09-16T11:45:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5250000000000002e-05 2024-09-16T11:45:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.603 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17730 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3768320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216356262000005, 2024-09-16T11:45:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030702365999999998, 2024-09-16T11:45:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17730 2024-09-16T11:45:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361305+0000", 2024-09-16T11:45:18.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:45:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361307+0000", 2024-09-16T11:45:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6020000000000001e-06 2024-09-16T11:45:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361324+0000", 2024-09-16T11:45:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7184999999999999e-05 2024-09-16T11:45:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361329+0000", 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391921+0000", 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03059222 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.391956+0000", 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4885999999999998e-05 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392007+0000", 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1718999999999997e-05 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17731 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 4087808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216328683, 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030771006, 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17731 2024-09-16T11:45:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361333+0000", 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361335+0000", 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.99e-06 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361343+0000", 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1869999999999997e-06 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361347+0000", 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392030+0000", 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030682831000000001 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392062+0000", 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2669000000000002e-05 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392104+0000", 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1323999999999999e-05 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.612 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17732 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2277376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216309988000006, 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030855441000000001, 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17732 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361351+0000", 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361352+0000", 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4000000000000002e-07 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361354+0000", 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7260000000000001e-06 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361360+0000", 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361363+0000", 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4630000000000001e-06 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392127+0000", 2024-09-16T11:45:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030763925000000001 2024-09-16T11:45:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392161+0000", 2024-09-16T11:45:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4017000000000003e-05 2024-09-16T11:45:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392207+0000", 2024-09-16T11:45:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5909999999999999e-05 2024-09-16T11:45:18.616 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17733 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216294978999997, 2024-09-16T11:45:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030925591999999998, 2024-09-16T11:45:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17733 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361366+0000", 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361367+0000", 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3999999999999997e-07 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361377+0000", 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0685e-05 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361384+0000", 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361392+0000", 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9500000000000001e-06 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392217+0000", 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030825657999999999 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392249+0000", 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1609999999999997e-05 2024-09-16T11:45:18.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392292+0000", 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3102000000000002e-05 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17734 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3022848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216262881000006, 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031015035, 2024-09-16T11:45:18.621 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17734 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361398+0000", 2024-09-16T11:45:18.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361399+0000", 2024-09-16T11:45:18.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:45:18.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361401+0000", 2024-09-16T11:45:18.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1519999999999999e-06 2024-09-16T11:45:18.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361408+0000", 2024-09-16T11:45:18.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1790000000000002e-06 2024-09-16T11:45:18.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361418+0000", 2024-09-16T11:45:18.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0023e-05 2024-09-16T11:45:18.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392316+0000", 2024-09-16T11:45:18.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030897153 2024-09-16T11:45:18.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392353+0000", 2024-09-16T11:45:18.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7577999999999998e-05 2024-09-16T11:45:18.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392414+0000", 2024-09-16T11:45:18.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0442999999999997e-05 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17735 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3129344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216239262000002, 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031087167999999998, 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.627 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17735 2024-09-16T11:45:18.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.627 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:18.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:18.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361422+0000", 2024-09-16T11:45:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:45:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361424+0000", 2024-09-16T11:45:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7990000000000001e-06 2024-09-16T11:45:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361430+0000", 2024-09-16T11:45:18.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7610000000000004e-06 2024-09-16T11:45:18.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361433+0000", 2024-09-16T11:45:18.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3050000000000001e-06 2024-09-16T11:45:18.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392426+0000", 2024-09-16T11:45:18.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.030992974 2024-09-16T11:45:18.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392461+0000", 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4962000000000002e-05 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392509+0000", 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7994999999999997e-05 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.630 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17736 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3448832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.631 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:18.631 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216224533000002, 2024-09-16T11:45:18.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031201058, 2024-09-16T11:45:18.631 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.631 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.631 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17736 2024-09-16T11:45:18.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.631 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361437+0000", 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:45:18.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361439+0000", 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5460000000000001e-06 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361451+0000", 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2472e-05 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361455+0000", 2024-09-16T11:45:18.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.551e-06 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392532+0000", 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031076989999999999 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392585+0000", 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3165999999999999e-05 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392638+0000", 2024-09-16T11:45:18.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2969000000000001e-05 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17737 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3874816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216202709000001, 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031293091000000002, 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.635 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17737 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361459+0000", 2024-09-16T11:45:18.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:45:18.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361461+0000", 2024-09-16T11:45:18.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8619999999999999e-06 2024-09-16T11:45:18.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361467+0000", 2024-09-16T11:45:18.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:45:18.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361470+0000", 2024-09-16T11:45:18.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:45:18.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392668+0000", 2024-09-16T11:45:18.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031197954999999999 2024-09-16T11:45:18.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392705+0000", 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7567999999999997e-05 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392752+0000", 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6347999999999997e-05 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17738 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2383872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:18.639 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216188059000004, 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031381778999999999, 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17738 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361473+0000", 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361474+0000", 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361475+0000", 2024-09-16T11:45:18.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361485+0000", 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361491+0000", 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392774+0000", 2024-09-16T11:45:18.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031283190000000002 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392810+0000", 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6427999999999997e-05 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392855+0000", 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4697000000000003e-05 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17739 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216166486000006, 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031460364999999997, 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17739 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:18.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:18.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361495+0000", 2024-09-16T11:45:18.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:45:18.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361498+0000", 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6450000000000001e-06 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361504+0000", 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361507+0000", 2024-09-16T11:45:18.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392877+0000", 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031369936000000001 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392911+0000", 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4193999999999998e-05 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392955+0000", 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3785000000000002e-05 2024-09-16T11:45:18.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17740 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216147039999996, 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.03153893, 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.648 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17740 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361514+0000", 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361515+0000", 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361516+0000", 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361522+0000", 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9689999999999999e-06 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361526+0000", 2024-09-16T11:45:18.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.392976+0000", 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031449593999999997 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393010+0000", 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4171999999999997e-05 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393053+0000", 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3531e-05 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17741 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3555328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216110342999997, 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031597482000000003, 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.652 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17741 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361551+0000", 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361552+0000", 2024-09-16T11:45:18.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-16T11:45:18.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361554+0000", 2024-09-16T11:45:18.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-16T11:45:18.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361565+0000", 2024-09-16T11:45:18.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1285e-05 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361569+0000", 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9659999999999998e-06 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393072+0000", 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031503273999999998 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393106+0000", 2024-09-16T11:45:18.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3516000000000002e-05 2024-09-16T11:45:18.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393149+0000", 2024-09-16T11:45:18.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2805000000000001e-05 2024-09-16T11:45:18.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.656 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.656 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17742 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3661824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.657 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:18.657 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216088396999993, 2024-09-16T11:45:18.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031675700000000001, 2024-09-16T11:45:18.657 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.657 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.657 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17742 2024-09-16T11:45:18.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.657 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361573+0000", 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361575+0000", 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361586+0000", 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1168999999999999e-05 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361591+0000", 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-16T11:45:18.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393170+0000", 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031578648000000001 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393205+0000", 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5737e-05 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393249+0000", 2024-09-16T11:45:18.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3241000000000002e-05 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17743 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3342336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216066510999994, 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031752835, 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17743 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361595+0000", 2024-09-16T11:45:18.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:45:18.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361597+0000", 2024-09-16T11:45:18.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.734e-06 2024-09-16T11:45:18.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361603+0000", 2024-09-16T11:45:18.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6520000000000003e-06 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361606+0000", 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.287e-06 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393270+0000", 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031663679 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393304+0000", 2024-09-16T11:45:18.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.447e-05 2024-09-16T11:45:18.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393348+0000", 2024-09-16T11:45:18.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3649999999999997e-05 2024-09-16T11:45:18.665 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.665 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.665 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.665 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:17744 4.1b 4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:e [read 3981312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:18.665 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.216051792000002, 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031837625000000001, 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 17744 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:18.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:18.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:18.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:18.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:18.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361610+0000", 2024-09-16T11:45:18.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:45:18.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361612+0000", 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.717e-06 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361619+0000", 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.361622+0000", 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:45:18.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393362+0000", 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031739898000000002 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393402+0000", 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9817000000000002e-05 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.393447+0000", 2024-09-16T11:45:18.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5410000000000001e-05 2024-09-16T11:45:18.670 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.670 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.670 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.670 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:18.670 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:18.670 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:18.672 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T11:45:18.672 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:45:18.672 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:18.670+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:18.772 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:18.771+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:18.810 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:18.819 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:45:18.903 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-16T11:45:18.902+0000 7fda827f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33321) UID: 0 2024-09-16T11:45:18.957 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:18.957 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:18.965 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:45:19.006 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-16T11:45:19.005+0000 7f82205f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33329) UID: 0 2024-09-16T11:45:19.075 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:19.073+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:19.104 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19943 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2416640~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:45:19.104 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.575164849000004, 2024-09-16T11:45:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027947257999999999, 2024-09-16T11:45:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19943 2024-09-16T11:45:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:45:19.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526295+0000", 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5999999999999999e-07 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526297+0000", 2024-09-16T11:45:19.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8059999999999999e-06 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526303+0000", 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6579999999999999e-06 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526306+0000", 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3969999999999998e-06 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554165+0000", 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027858519000000002 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554201+0000", 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5908000000000002e-05 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554242+0000", 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1510000000000001e-05 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19944 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2842624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.575149953999997, 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028052639000000001, 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19944 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.109 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526310+0000", 2024-09-16T11:45:19.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526312+0000", 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526317+0000", 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5910000000000003e-06 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526321+0000", 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2339999999999999e-06 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554263+0000", 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027942181 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554307+0000", 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4499999999999997e-05 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554362+0000", 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5078999999999999e-05 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19945 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3162112~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.575135399000004, 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028145673999999999, 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.113 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19945 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526324+0000", 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526325+0000", 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526326+0000", 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.733e-06 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526332+0000", 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526336+0000", 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8410000000000002e-06 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554389+0000", 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028053576 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554427+0000", 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7299999999999999e-05 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554470+0000", 2024-09-16T11:45:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3195000000000002e-05 2024-09-16T11:45:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19946 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3268608~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:45:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.575119819999998, 2024-09-16T11:45:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028232587, 2024-09-16T11:45:19.117 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19946 2024-09-16T11:45:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:45:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:45:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526340+0000", 2024-09-16T11:45:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:45:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526346+0000", 2024-09-16T11:45:19.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6459999999999998e-06 2024-09-16T11:45:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526352+0000", 2024-09-16T11:45:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1070000000000001e-06 2024-09-16T11:45:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526355+0000", 2024-09-16T11:45:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:45:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554493+0000", 2024-09-16T11:45:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.02813746 2024-09-16T11:45:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554530+0000", 2024-09-16T11:45:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6881999999999997e-05 2024-09-16T11:45:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554572+0000", 2024-09-16T11:45:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2688999999999997e-05 2024-09-16T11:45:19.121 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19947 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4014080~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:45:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.575100465999995, 2024-09-16T11:45:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028315646999999999, 2024-09-16T11:45:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19947 2024-09-16T11:45:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:45:19.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526359+0000", 2024-09-16T11:45:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526360+0000", 2024-09-16T11:45:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.01e-07 2024-09-16T11:45:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526361+0000", 2024-09-16T11:45:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8300000000000001e-06 2024-09-16T11:45:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526369+0000", 2024-09-16T11:45:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:45:19.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526377+0000", 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0909999999999993e-06 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554593+0000", 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028216657999999999 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554632+0000", 2024-09-16T11:45:19.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8828000000000002e-05 2024-09-16T11:45:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554675+0000", 2024-09-16T11:45:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2710000000000003e-05 2024-09-16T11:45:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19948 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3588096~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:45:19.127 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.575078067000007, 2024-09-16T11:45:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028394545, 2024-09-16T11:45:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19948 2024-09-16T11:45:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:45:19.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:45:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526382+0000", 2024-09-16T11:45:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9200000000000001e-07 2024-09-16T11:45:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526384+0000", 2024-09-16T11:45:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:45:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526391+0000", 2024-09-16T11:45:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5150000000000003e-06 2024-09-16T11:45:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526394+0000", 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4240000000000002e-06 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554695+0000", 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028301218999999999 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554731+0000", 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6298999999999999e-05 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554776+0000", 2024-09-16T11:45:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4675999999999997e-05 2024-09-16T11:45:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19949 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1351680~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:45:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.575061794999996, 2024-09-16T11:45:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028490081, 2024-09-16T11:45:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19949 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:45:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526398+0000", 2024-09-16T11:45:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:45:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526400+0000", 2024-09-16T11:45:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7099999999999999e-06 2024-09-16T11:45:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526406+0000", 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5140000000000001e-06 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526409+0000", 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5360000000000001e-06 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554801+0000", 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028392378999999999 2024-09-16T11:45:19.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554843+0000", 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1056000000000001e-05 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554888+0000", 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466999999999997e-05 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19950 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 1777664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:45:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.575046799000006, 2024-09-16T11:45:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028661844999999998, 2024-09-16T11:45:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19950 2024-09-16T11:45:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526413+0000", 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-16T11:45:19.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526415+0000", 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.776e-06 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526422+0000", 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6930000000000001e-06 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526425+0000", 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.554923+0000", 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028498102000000001 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555000+0000", 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6619999999999995e-05 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555075+0000", 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4931999999999993e-05 2024-09-16T11:45:19.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19951 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2097152~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.575030987999995, 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028755718, 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19951 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526429+0000", 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526431+0000", 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526441+0000", 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7790000000000007e-06 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526445+0000", 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.634e-06 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555100+0000", 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654722000000001 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555138+0000", 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7996e-05 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555184+0000", 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6427999999999997e-05 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19952 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2203648~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.575010427999999, 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028901157, 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19952 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526449+0000", 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526450+0000", 2024-09-16T11:45:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:45:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526452+0000", 2024-09-16T11:45:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9039999999999999e-06 2024-09-16T11:45:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526465+0000", 2024-09-16T11:45:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3559e-05 2024-09-16T11:45:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526470+0000", 2024-09-16T11:45:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7110000000000003e-06 2024-09-16T11:45:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555217+0000", 2024-09-16T11:45:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028747287999999999 2024-09-16T11:45:19.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555272+0000", 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4885000000000001e-05 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555350+0000", 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8428000000000002e-05 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19953 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2523136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.574985835999996, 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029004309999999998, 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19953 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526474+0000", 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526476+0000", 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9810000000000002e-06 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526482+0000", 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7740000000000001e-06 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526485+0000", 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2760000000000001e-06 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555383+0000", 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028898113 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555433+0000", 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9753999999999998e-05 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555478+0000", 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5006999999999998e-05 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19954 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 2949120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.574970839000002, 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029165054999999999, 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19954 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526489+0000", 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1799999999999995e-07 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526491+0000", 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7320000000000001e-06 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526497+0000", 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6869999999999999e-06 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526500+0000", 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:45:19.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555513+0000", 2024-09-16T11:45:19.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029012626 2024-09-16T11:45:19.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555576+0000", 2024-09-16T11:45:19.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3288999999999997e-05 2024-09-16T11:45:19.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555654+0000", 2024-09-16T11:45:19.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7861999999999994e-05 2024-09-16T11:45:19.158 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.158 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.158 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.158 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19955 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3375104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.158 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:45:19.158 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.574951537000004, 2024-09-16T11:45:19.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029241979000000001, 2024-09-16T11:45:19.158 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.158 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.159 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19955 2024-09-16T11:45:19.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.159 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:45:19.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526508+0000", 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526509+0000", 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.65e-07 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526510+0000", 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.844e-06 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526516+0000", 2024-09-16T11:45:19.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9039999999999997e-06 2024-09-16T11:45:19.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526520+0000", 2024-09-16T11:45:19.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3639999999999999e-06 2024-09-16T11:45:19.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555669+0000", 2024-09-16T11:45:19.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029149628 2024-09-16T11:45:19.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555706+0000", 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7017000000000002e-05 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555750+0000", 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3856999999999997e-05 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.163 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19956 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 3694592~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.163 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:45:19.163 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.574936582000007, 2024-09-16T11:45:19.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029398799999999999, 2024-09-16T11:45:19.163 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.163 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.163 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19956 2024-09-16T11:45:19.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.163 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526523+0000", 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526524+0000", 2024-09-16T11:45:19.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526525+0000", 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526531+0000", 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6010000000000002e-06 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526535+0000", 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.597e-06 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555785+0000", 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029250235999999999 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555845+0000", 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0393000000000003e-05 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555922+0000", 2024-09-16T11:45:19.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6710000000000002e-05 2024-09-16T11:45:19.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.167 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.167 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19957 4.8 4:104aaeaf:::rbd_data.11c1a7dbfac1.0000000000000006:f [read 4120576~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.167 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:45:19.167 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.574921429, 2024-09-16T11:45:19.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029481157000000001, 2024-09-16T11:45:19.167 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.167 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.168 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19957 2024-09-16T11:45:19.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.168 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:45:19.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526538+0000", 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526539+0000", 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526540+0000", 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.759e-06 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526546+0000", 2024-09-16T11:45:19.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1079999999999998e-06 2024-09-16T11:45:19.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.526550+0000", 2024-09-16T11:45:19.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.269e-06 2024-09-16T11:45:19.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555939+0000", 2024-09-16T11:45:19.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029389444000000001 2024-09-16T11:45:19.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.555977+0000", 2024-09-16T11:45:19.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7487999999999998e-05 2024-09-16T11:45:19.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.556019+0000", 2024-09-16T11:45:19.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2756999999999997e-05 2024-09-16T11:45:19.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.171 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:28959 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.092065016999996, 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031145285000000002, 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 28959 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.172 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:45:19.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009395+0000", 2024-09-16T11:45:19.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.009396+0000", 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.296e-06 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030864+0000", 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.021468451 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030892+0000", 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7555999999999998e-05 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030902+0000", 2024-09-16T11:45:19.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9550000000000004e-06 2024-09-16T11:45:19.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.030979+0000", 2024-09-16T11:45:19.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7137000000000004e-05 2024-09-16T11:45:19.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031051+0000", 2024-09-16T11:45:19.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1475000000000003e-05 2024-09-16T11:45:19.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:45:19.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031389+0000", 2024-09-16T11:45:19.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033827100000000001 2024-09-16T11:45:19.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:19.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.039264+0000", 2024-09-16T11:45:19.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078751700000000008 2024-09-16T11:45:19.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:19.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040484+0000", 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012199579999999999 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040517+0000", 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2960999999999998e-05 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040540+0000", 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3054999999999999e-05 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.177 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30605 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 3915776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.857758758000003, 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938655999999999, 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30605 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.178 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243701+0000", 2024-09-16T11:45:19.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243704+0000", 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2199999999999999e-06 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243710+0000", 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6340000000000003e-06 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243714+0000", 2024-09-16T11:45:19.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5030000000000002e-06 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271450+0000", 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027736035999999999 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271536+0000", 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6521999999999996e-05 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271640+0000", 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103341 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30607 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4046848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.857735602999995, 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028130446, 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.183 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30607 2024-09-16T11:45:19.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.183 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:45:19.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243724+0000", 2024-09-16T11:45:19.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243725+0000", 2024-09-16T11:45:19.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2399999999999998e-07 2024-09-16T11:45:19.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243727+0000", 2024-09-16T11:45:19.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3379999999999999e-06 2024-09-16T11:45:19.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243733+0000", 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4280000000000001e-06 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243737+0000", 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8990000000000003e-06 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271676+0000", 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027938481000000001 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271757+0000", 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1396999999999993e-05 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271855+0000", 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7379000000000004e-05 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:30608 4.8 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:16 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.857718751999997, 2024-09-16T11:45:19.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028323879, 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30608 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:45:19.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243741+0000", 2024-09-16T11:45:19.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:45:19.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243743+0000", 2024-09-16T11:45:19.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.762e-06 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243749+0000", 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5670000000000002e-06 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.243758+0000", 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8309999999999996e-06 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271879+0000", 2024-09-16T11:45:19.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028120501999999999 2024-09-16T11:45:19.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.271965+0000", 2024-09-16T11:45:19.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5755999999999997e-05 2024-09-16T11:45:19.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.272065+0000", 2024-09-16T11:45:19.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000100101 2024-09-16T11:45:19.190 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.190 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.191 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.191 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:56045 4.15 4:ab2ce1b7:::rbd_data.11c1b01b799a.0000000000000020:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-16T11:45:19.191 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:45:19.191 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.922435501000002, 2024-09-16T11:45:19.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032248165000000002, 2024-09-16T11:45:19.191 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.191 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:19.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 56045 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179024+0000", 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179026+0000", 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.556e-06 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179032+0000", 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5379999999999999e-06 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179045+0000", 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.277e-05 2024-09-16T11:45:19.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179067+0000", 2024-09-16T11:45:19.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.243e-05 2024-09-16T11:45:19.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179120+0000", 2024-09-16T11:45:19.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2726000000000003e-05 2024-09-16T11:45:19.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.179279+0000", 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015865899999999999 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.185163+0000", 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0058837639999999997 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211197+0000", 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026034130999999999 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211206+0000", 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2539999999999998e-06 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211245+0000", 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8924999999999997e-05 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:29.211272+0000", 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7412000000000001e-05 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.196 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.197 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.197 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.197 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.197 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:19.198 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:45:19.199 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-16T11:45:19.197+0000 7fd33f7f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33323) UID: 0 2024-09-16T11:45:19.299 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:19.297+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:19.311 INFO:teuthology.orchestra.run.smithi070.stdout: marking collection for removal 2024-09-16T11:45:19.312 INFO:teuthology.orchestra.run.smithi070.stdout:setting '_remove' omap key 2024-09-16T11:45:19.318 INFO:teuthology.orchestra.run.smithi070.stdout:finish_remove_pgs 4.1b_head removing 4.1b 2024-09-16T11:45:19.322 INFO:teuthology.orchestra.run.smithi070.stdout:Remove successful 2024-09-16T11:45:19.336 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:19.337 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:19.337 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:19.345 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:45:19.400 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:19.398+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:19.442 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:19.443 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:19.443 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:19.443 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:19.443 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:19.451 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:45:19.535 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-16T11:45:19.534+0000 7f0f543f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33326) UID: 0 2024-09-16T11:45:19.591 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:19.591 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:19.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:19.591 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:19.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.591 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19282 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7127040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.591 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:45:19.592 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.115727254000006, 2024-09-16T11:45:19.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521151000000002, 2024-09-16T11:45:19.592 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.592 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.592 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.592 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.592 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.592 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19282 2024-09-16T11:45:19.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.592 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:45:19.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472837+0000", 2024-09-16T11:45:19.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472838+0000", 2024-09-16T11:45:19.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:45:19.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472848+0000", 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9799999999999993e-06 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472858+0000", 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9429999999999995e-06 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.472862+0000", 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4460000000000003e-06 2024-09-16T11:45:19.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501277+0000", 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028414900999999999 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501322+0000", 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4880999999999999e-05 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501358+0000", 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6245000000000003e-05 2024-09-16T11:45:19.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:19286 4.1f 4:fbc4c982:::rbd_data.11c1a7dbfac1.0000000000000003:f [read 7979008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.115354322000002, 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028521860999999999, 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19286 2024-09-16T11:45:19.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473210+0000", 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473211+0000", 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473214+0000", 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1080000000000001e-06 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473226+0000", 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2441e-05 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.473234+0000", 2024-09-16T11:45:19.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729999999999998e-06 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501592+0000", 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028358491999999999 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501671+0000", 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8846999999999999e-05 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.501732+0000", 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1118000000000007e-05 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20619 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7610368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.028005153999999, 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028500152000000001, 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20619 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560559+0000", 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560560+0000", 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9499999999999998e-07 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560561+0000", 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.071e-06 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560565+0000", 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:45:19.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560567+0000", 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588949+0000", 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028381878999999999 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.588989+0000", 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0649000000000003e-05 2024-09-16T11:45:19.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589059+0000", 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0239000000000003e-05 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:20620 4.5 4:a5d86847:::rbd_data.11c11284e6d8.0000000000000000:e [read 7716864~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.027995754000003, 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028627621999999998, 2024-09-16T11:45:19.604 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.605 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.605 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20620 2024-09-16T11:45:19.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.605 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:45:19.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.560569+0000", 2024-09-16T11:45:19.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562377+0000", 2024-09-16T11:45:19.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018081729999999999 2024-09-16T11:45:19.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562380+0000", 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0469999999999998e-06 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562386+0000", 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2040000000000002e-06 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.562390+0000", 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589092+0000", 2024-09-16T11:45:19.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026701798999999998 2024-09-16T11:45:19.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589139+0000", 2024-09-16T11:45:19.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7320000000000001e-05 2024-09-16T11:45:19.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:55.589196+0000", 2024-09-16T11:45:19.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7201999999999999e-05 2024-09-16T11:45:19.608 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.609 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.609 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41954 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.609 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:45:19.609 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174289963999996, 2024-09-16T11:45:19.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028594441000000002, 2024-09-16T11:45:19.609 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.609 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.610 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41954 2024-09-16T11:45:19.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.610 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:45:19.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414274+0000", 2024-09-16T11:45:19.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414275+0000", 2024-09-16T11:45:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:45:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414277+0000", 2024-09-16T11:45:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.852e-06 2024-09-16T11:45:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414282+0000", 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7559999999999996e-06 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414286+0000", 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442778+0000", 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028492237 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442816+0000", 2024-09-16T11:45:19.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8198000000000003e-05 2024-09-16T11:45:19.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442869+0000", 2024-09-16T11:45:19.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2589000000000001e-05 2024-09-16T11:45:19.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.613 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.613 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41955 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174274905999994, 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028687865, 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41955 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414290+0000", 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414292+0000", 2024-09-16T11:45:19.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.81e-06 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414299+0000", 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4279999999999999e-06 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414303+0000", 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442892+0000", 2024-09-16T11:45:19.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028589564000000001 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442930+0000", 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7330999999999999e-05 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.442977+0000", 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7874000000000003e-05 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41956 4.5 4:a12bfdab:::rbd_data.11c1bd2a98f3.0000000000000003:1a [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174239588999995, 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028766896, 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41956 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:45:19.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414325+0000", 2024-09-16T11:45:19.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414326+0000", 2024-09-16T11:45:19.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9900000000000009e-07 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414330+0000", 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2339999999999996e-06 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414341+0000", 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1358999999999999e-05 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414347+0000", 2024-09-16T11:45:19.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8100000000000003e-06 2024-09-16T11:45:19.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443002+0000", 2024-09-16T11:45:19.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028654331000000002 2024-09-16T11:45:19.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443040+0000", 2024-09-16T11:45:19.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8262000000000001e-05 2024-09-16T11:45:19.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443092+0000", 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1901000000000003e-05 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41957 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174212170000004, 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028848318000000001, 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.622 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.623 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41957 2024-09-16T11:45:19.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.623 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:45:19.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414352+0000", 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414353+0000", 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414355+0000", 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3489999999999999e-06 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414364+0000", 2024-09-16T11:45:19.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1449999999999997e-06 2024-09-16T11:45:19.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414368+0000", 2024-09-16T11:45:19.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8940000000000003e-06 2024-09-16T11:45:19.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443115+0000", 2024-09-16T11:45:19.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028746532000000002 2024-09-16T11:45:19.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443152+0000", 2024-09-16T11:45:19.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6814999999999999e-05 2024-09-16T11:45:19.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443201+0000", 2024-09-16T11:45:19.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9069000000000001e-05 2024-09-16T11:45:19.626 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.626 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.626 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.627 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41958 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.627 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:45:19.627 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174188388000005, 2024-09-16T11:45:19.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028939856, 2024-09-16T11:45:19.627 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.627 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.627 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41958 2024-09-16T11:45:19.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.628 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:45:19.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414376+0000", 2024-09-16T11:45:19.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414377+0000", 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4099999999999999e-07 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414379+0000", 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.418e-06 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414385+0000", 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999999e-06 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414389+0000", 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7740000000000002e-06 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443223+0000", 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028833919 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443263+0000", 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0148999999999998e-05 2024-09-16T11:45:19.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443316+0000", 2024-09-16T11:45:19.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2636000000000002e-05 2024-09-16T11:45:19.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.631 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.631 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41959 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.631 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:19.631 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174171462999993, 2024-09-16T11:45:19.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029040309, 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41959 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414393+0000", 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414395+0000", 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7579999999999999e-06 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414401+0000", 2024-09-16T11:45:19.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5330000000000002e-06 2024-09-16T11:45:19.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414404+0000", 2024-09-16T11:45:19.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.529e-06 2024-09-16T11:45:19.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443339+0000", 2024-09-16T11:45:19.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028934933999999999 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443383+0000", 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3832000000000003e-05 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443433+0000", 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0309000000000002e-05 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41960 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174156628999995, 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029138134, 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41960 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414408+0000", 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-07 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414410+0000", 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.756e-06 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414417+0000", 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414420+0000", 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:45:19.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443457+0000", 2024-09-16T11:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029037065000000001 2024-09-16T11:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443495+0000", 2024-09-16T11:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7555000000000002e-05 2024-09-16T11:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443546+0000", 2024-09-16T11:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1422000000000003e-05 2024-09-16T11:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41961 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174140648000005, 2024-09-16T11:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029230973, 2024-09-16T11:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41961 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:19.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414424+0000", 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4499999999999998e-07 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414432+0000", 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.413e-06 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414439+0000", 2024-09-16T11:45:19.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4769999999999997e-06 2024-09-16T11:45:19.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414443+0000", 2024-09-16T11:45:19.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8859999999999997e-06 2024-09-16T11:45:19.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443569+0000", 2024-09-16T11:45:19.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029126124 2024-09-16T11:45:19.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443606+0000", 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6588999999999998e-05 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443655+0000", 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9138999999999998e-05 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.644 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41962 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.645 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:19.645 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174117680999998, 2024-09-16T11:45:19.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029324505000000001, 2024-09-16T11:45:19.645 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.645 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.645 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41962 2024-09-16T11:45:19.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.645 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414447+0000", 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414449+0000", 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0279999999999999e-06 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414456+0000", 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3339999999999998e-06 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414459+0000", 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.382e-06 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443678+0000", 2024-09-16T11:45:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029218958999999999 2024-09-16T11:45:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443719+0000", 2024-09-16T11:45:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0722000000000001e-05 2024-09-16T11:45:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443771+0000", 2024-09-16T11:45:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2661000000000003e-05 2024-09-16T11:45:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.648 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41963 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174101777999994, 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029416557999999999, 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41963 2024-09-16T11:45:19.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414463+0000", 2024-09-16T11:45:19.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414465+0000", 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414470+0000", 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4750000000000001e-06 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414474+0000", 2024-09-16T11:45:19.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3720000000000001e-06 2024-09-16T11:45:19.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443795+0000", 2024-09-16T11:45:19.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029320821 2024-09-16T11:45:19.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443830+0000", 2024-09-16T11:45:19.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5463000000000002e-05 2024-09-16T11:45:19.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443879+0000", 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9147000000000002e-05 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41964 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174086889999998, 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029521202999999999, 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41964 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414478+0000", 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414480+0000", 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414486+0000", 2024-09-16T11:45:19.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:45:19.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414489+0000", 2024-09-16T11:45:19.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2770000000000001e-06 2024-09-16T11:45:19.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443903+0000", 2024-09-16T11:45:19.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029414109000000001 2024-09-16T11:45:19.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443944+0000", 2024-09-16T11:45:19.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0992999999999999e-05 2024-09-16T11:45:19.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.443999+0000", 2024-09-16T11:45:19.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4632000000000001e-05 2024-09-16T11:45:19.657 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.657 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.657 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41965 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174071928000004, 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029611293, 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41965 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414493+0000", 2024-09-16T11:45:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:45:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414495+0000", 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.88e-06 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414501+0000", 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2489999999999998e-06 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414504+0000", 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3139999999999999e-06 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444023+0000", 2024-09-16T11:45:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029518564000000001 2024-09-16T11:45:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444058+0000", 2024-09-16T11:45:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5196e-05 2024-09-16T11:45:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444104+0000", 2024-09-16T11:45:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5732999999999997e-05 2024-09-16T11:45:19.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41966 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174055722000006, 2024-09-16T11:45:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029707678000000001, 2024-09-16T11:45:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41966 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414509+0000", 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414514+0000", 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414518+0000", 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3839999999999999e-06 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414521+0000", 2024-09-16T11:45:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0400000000000001e-06 2024-09-16T11:45:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444128+0000", 2024-09-16T11:45:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029606612000000001 2024-09-16T11:45:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444167+0000", 2024-09-16T11:45:19.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9418000000000001e-05 2024-09-16T11:45:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444216+0000", 2024-09-16T11:45:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9354999999999998e-05 2024-09-16T11:45:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41967 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174039913000001, 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029796916, 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41967 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414525+0000", 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6699999999999999e-07 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414527+0000", 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7060000000000001e-06 2024-09-16T11:45:19.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414533+0000", 2024-09-16T11:45:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2140000000000001e-06 2024-09-16T11:45:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414536+0000", 2024-09-16T11:45:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2229999999999999e-06 2024-09-16T11:45:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444239+0000", 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029702698999999999 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444273+0000", 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4229e-05 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444321+0000", 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8477999999999999e-05 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41968 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174025016000002, 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029902765000000001, 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41968 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:45:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414539+0000", 2024-09-16T11:45:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414540+0000", 2024-09-16T11:45:19.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:45:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414542+0000", 2024-09-16T11:45:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.798e-06 2024-09-16T11:45:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414547+0000", 2024-09-16T11:45:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4639999999999997e-06 2024-09-16T11:45:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414551+0000", 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6279999999999998e-06 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444346+0000", 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029794885 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444389+0000", 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3442000000000002e-05 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444442+0000", 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3173000000000001e-05 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:41969 4.5 4:a7887a37:::rbd_data.11c1bd2a98f3.0000000000000004:1a [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.174010104000004, 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029997612999999999, 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-16T11:45:19.675 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 41969 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414554+0000", 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:19.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414555+0000", 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414556+0000", 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.725e-06 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414563+0000", 2024-09-16T11:45:19.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2530000000000001e-06 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.414566+0000", 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4429999999999999e-06 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444463+0000", 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029897271 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444503+0000", 2024-09-16T11:45:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9782e-05 2024-09-16T11:45:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:06.444552+0000", 2024-09-16T11:45:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8789000000000003e-05 2024-09-16T11:45:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:19.679 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:19.679 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:19.680 INFO:teuthology.orchestra.run.smithi070.stdout:All operations completed A-OK! 2024-09-16T11:45:19.681 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:45:19.681 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-16T11:45:19.678+0000 7f5fb43f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33334) UID: 0 2024-09-16T11:45:19.688 DEBUG:teuthology.parallel:result is None 2024-09-16T11:45:19.688 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-16T11:45:19.698 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-16T11:45:19.733 INFO:tasks.thrashosds:joining thrashosds 2024-09-16T11:45:19.828 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.4.1b.3 2024-09-16T11:45:19.831 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-16T11:45:19.831 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:19.839 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:45:19.937 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:19.937 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-16T11:45:19.937 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-16T11:45:19.937 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-16T11:45:19.937 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:19.945 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:45:20.044 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-16T11:45:20.044 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-16T11:45:20.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-16T11:45:20.044 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-16T11:45:20.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.044 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:20.044 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:45:20.045 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 127.121256864, 2024-09-16T11:45:20.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071531770000000001, 2024-09-16T11:45:20.045 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.045 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.045 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:20.045 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:20.045 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:20.045 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-16T11:45:20.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.045 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:45:20.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920069+0000", 2024-09-16T11:45:20.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920070+0000", 2024-09-16T11:45:20.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:45:20.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920112+0000", 2024-09-16T11:45:20.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2383999999999999e-05 2024-09-16T11:45:20.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920119+0000", 2024-09-16T11:45:20.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.224e-06 2024-09-16T11:45:20.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920122+0000", 2024-09-16T11:45:20.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0249999999999998e-06 2024-09-16T11:45:20.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924086+0000", 2024-09-16T11:45:20.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039648740000000002 2024-09-16T11:45:20.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924139+0000", 2024-09-16T11:45:20.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2750999999999997e-05 2024-09-16T11:45:20.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:20.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924219+0000", 2024-09-16T11:45:20.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0217999999999997e-05 2024-09-16T11:45:20.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:20.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925669+0000", 2024-09-16T11:45:20.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014495759999999999 2024-09-16T11:45:20.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:20.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927176+0000", 2024-09-16T11:45:20.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015066809999999999 2024-09-16T11:45:20.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927199+0000", 2024-09-16T11:45:20.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3006999999999998e-05 2024-09-16T11:45:20.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927223+0000", 2024-09-16T11:45:20.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3859999999999999e-05 2024-09-16T11:45:20.050 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.050 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.050 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 127.121199934, 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071491640000000004, 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:45:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920126+0000", 2024-09-16T11:45:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920127+0000", 2024-09-16T11:45:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920129+0000", 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1560000000000002e-06 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920145+0000", 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5787999999999999e-05 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920147+0000", 2024-09-16T11:45:20.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-16T11:45:20.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924878+0000", 2024-09-16T11:45:20.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047310820000000002 2024-09-16T11:45:20.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.924927+0000", 2024-09-16T11:45:20.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8294000000000002e-05 2024-09-16T11:45:20.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:20.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925006+0000", 2024-09-16T11:45:20.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8986000000000006e-05 2024-09-16T11:45:20.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:20.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926700+0000", 2024-09-16T11:45:20.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016939870000000001 2024-09-16T11:45:20.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:20.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927230+0000", 2024-09-16T11:45:20.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053063299999999997 2024-09-16T11:45:20.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927253+0000", 2024-09-16T11:45:20.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3025000000000001e-05 2024-09-16T11:45:20.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927275+0000", 2024-09-16T11:45:20.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2150999999999998e-05 2024-09-16T11:45:20.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.056 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 127.12117456599999, 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071775449999999996, 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.057 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920152+0000", 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5699999999999999e-07 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920160+0000", 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920182+0000", 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1919000000000001e-05 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920187+0000", 2024-09-16T11:45:20.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-16T11:45:20.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925276+0000", 2024-09-16T11:45:20.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050891390000000003 2024-09-16T11:45:20.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925324+0000", 2024-09-16T11:45:20.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7644e-05 2024-09-16T11:45:20.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:20.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925413+0000", 2024-09-16T11:45:20.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9227999999999993e-05 2024-09-16T11:45:20.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:20.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926735+0000", 2024-09-16T11:45:20.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013225540000000001 2024-09-16T11:45:20.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:20.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927283+0000", 2024-09-16T11:45:20.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00054737799999999995 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927304+0000", 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1563e-05 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927329+0000", 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4967e-05 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.063 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:20.063 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:20.063 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 127.121122598, 2024-09-16T11:45:20.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007233639, 2024-09-16T11:45:20.063 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.063 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:20.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:20.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:20.063 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-16T11:45:20.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.063 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:20.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:20.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920204+0000", 2024-09-16T11:45:20.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-16T11:45:20.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920254+0000", 2024-09-16T11:45:20.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0074999999999997e-05 2024-09-16T11:45:20.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920264+0000", 2024-09-16T11:45:20.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9680000000000001e-06 2024-09-16T11:45:20.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920270+0000", 2024-09-16T11:45:20.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1769999999999997e-06 2024-09-16T11:45:20.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925696+0000", 2024-09-16T11:45:20.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054262679999999997 2024-09-16T11:45:20.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925751+0000", 2024-09-16T11:45:20.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5114000000000001e-05 2024-09-16T11:45:20.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:20.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.925833+0000", 2024-09-16T11:45:20.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1596999999999998e-05 2024-09-16T11:45:20.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:20.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927338+0000", 2024-09-16T11:45:20.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505665 2024-09-16T11:45:20.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:20.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927387+0000", 2024-09-16T11:45:20.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8497e-05 2024-09-16T11:45:20.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927414+0000", 2024-09-16T11:45:20.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7396999999999999e-05 2024-09-16T11:45:20.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927437+0000", 2024-09-16T11:45:20.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3164e-05 2024-09-16T11:45:20.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.069 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.069 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:20.069 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:20.069 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 127.121041488, 2024-09-16T11:45:20.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097403620000000007, 2024-09-16T11:45:20.069 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:20.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920285+0000", 2024-09-16T11:45:20.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2699999999999999e-07 2024-09-16T11:45:20.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920289+0000", 2024-09-16T11:45:20.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:45:20.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920296+0000", 2024-09-16T11:45:20.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1899999999999998e-06 2024-09-16T11:45:20.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920304+0000", 2024-09-16T11:45:20.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4549999999999998e-06 2024-09-16T11:45:20.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926241+0000", 2024-09-16T11:45:20.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059373760000000003 2024-09-16T11:45:20.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926292+0000", 2024-09-16T11:45:20.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1149000000000001e-05 2024-09-16T11:45:20.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:20.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926398+0000", 2024-09-16T11:45:20.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105202 2024-09-16T11:45:20.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:20.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927468+0000", 2024-09-16T11:45:20.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00106982 2024-09-16T11:45:20.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:20.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.929966+0000", 2024-09-16T11:45:20.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024988620000000001 2024-09-16T11:45:20.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930003+0000", 2024-09-16T11:45:20.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6405999999999997e-05 2024-09-16T11:45:20.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930025+0000", 2024-09-16T11:45:20.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2365e-05 2024-09-16T11:45:20.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.075 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.076 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:20.076 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:20.076 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 127.12101551000001, 2024-09-16T11:45:20.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0099929170000000005, 2024-09-16T11:45:20.076 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.076 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:20.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:20.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:20.076 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-16T11:45:20.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.076 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:20.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:20.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920311+0000", 2024-09-16T11:45:20.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:45:20.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920316+0000", 2024-09-16T11:45:20.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.33e-06 2024-09-16T11:45:20.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920322+0000", 2024-09-16T11:45:20.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6649999999999997e-06 2024-09-16T11:45:20.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.920326+0000", 2024-09-16T11:45:20.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9430000000000002e-06 2024-09-16T11:45:20.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926756+0000", 2024-09-16T11:45:20.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064300290000000003 2024-09-16T11:45:20.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926807+0000", 2024-09-16T11:45:20.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0648e-05 2024-09-16T11:45:20.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:20.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.926893+0000", 2024-09-16T11:45:20.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6372999999999994e-05 2024-09-16T11:45:20.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:20.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.927606+0000", 2024-09-16T11:45:20.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000712581 2024-09-16T11:45:20.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:20.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930260+0000", 2024-09-16T11:45:20.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026545599999999998 2024-09-16T11:45:20.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930286+0000", 2024-09-16T11:45:20.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6035999999999999e-05 2024-09-16T11:45:20.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:12.930304+0000", 2024-09-16T11:45:20.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7258e-05 2024-09-16T11:45:20.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 126.70349688500001, 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082931280000000003, 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:20.082 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-16T11:45:20.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.083 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:45:20.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337829+0000", 2024-09-16T11:45:20.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337830+0000", 2024-09-16T11:45:20.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:45:20.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337833+0000", 2024-09-16T11:45:20.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:45:20.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337840+0000", 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.285e-06 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337846+0000", 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4999999999999996e-06 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343568+0000", 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057219580000000001 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343621+0000", 2024-09-16T11:45:20.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2611000000000002e-05 2024-09-16T11:45:20.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:20.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.343693+0000", 2024-09-16T11:45:20.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2731999999999994e-05 2024-09-16T11:45:20.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:20.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344988+0000", 2024-09-16T11:45:20.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012944759999999999 2024-09-16T11:45:20.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346075+0000", 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010875349999999999 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346102+0000", 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6318000000000001e-05 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346123+0000", 2024-09-16T11:45:20.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0812e-05 2024-09-16T11:45:20.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.088 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.088 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:20.088 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:45:20.088 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 126.703474954, 2024-09-16T11:45:20.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085533580000000005, 2024-09-16T11:45:20.088 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.088 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:20.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:20.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:20.089 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-16T11:45:20.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.089 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:45:20.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337851+0000", 2024-09-16T11:45:20.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337852+0000", 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2800000000000002e-07 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337908+0000", 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6045999999999998e-05 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337916+0000", 2024-09-16T11:45:20.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-16T11:45:20.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337921+0000", 2024-09-16T11:45:20.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5460000000000002e-06 2024-09-16T11:45:20.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344142+0000", 2024-09-16T11:45:20.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062211080000000004 2024-09-16T11:45:20.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344186+0000", 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4446000000000001e-05 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344257+0000", 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0816999999999996e-05 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345180+0000", 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000923042 2024-09-16T11:45:20.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:20.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346358+0000", 2024-09-16T11:45:20.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011780040000000001 2024-09-16T11:45:20.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346388+0000", 2024-09-16T11:45:20.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9470000000000001e-05 2024-09-16T11:45:20.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346405+0000", 2024-09-16T11:45:20.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7062e-05 2024-09-16T11:45:20.094 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.094 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.094 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.094 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4181.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-16T11:45:20.094 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:45:20.094 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 126.70339806600001, 2024-09-16T11:45:20.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085252580000000008, 2024-09-16T11:45:20.094 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.094 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:20.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4181", 2024-09-16T11:45:20.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/30286296", 2024-09-16T11:45:20.095 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-16T11:45:20.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.095 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:45:20.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337928+0000", 2024-09-16T11:45:20.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337929+0000", 2024-09-16T11:45:20.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.44e-07 2024-09-16T11:45:20.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337932+0000", 2024-09-16T11:45:20.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5650000000000002e-06 2024-09-16T11:45:20.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337939+0000", 2024-09-16T11:45:20.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3069999999999999e-06 2024-09-16T11:45:20.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.337942+0000", 2024-09-16T11:45:20.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-16T11:45:20.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344638+0000", 2024-09-16T11:45:20.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066959869999999996 2024-09-16T11:45:20.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344683+0000", 2024-09-16T11:45:20.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4515999999999999e-05 2024-09-16T11:45:20.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-16T11:45:20.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.344758+0000", 2024-09-16T11:45:20.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4851000000000006e-05 2024-09-16T11:45:20.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.345392+0000", 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063416000000000004 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346412+0000", 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001020242 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346432+0000", 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0284000000000001e-05 2024-09-16T11:45:20.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:43:13.346453+0000", 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000001e-05 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28187 4.17 e52/28 4:ee916bd9:::rbd_data.11c1ce740b46.0000000000000000:head v 52'195, mlcod=52'193)", 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 79.531978679999995, 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075854429999999999, 2024-09-16T11:45:20.100 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.101 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.101 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:45:20.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509348+0000", 2024-09-16T11:45:20.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.509349+0000", 2024-09-16T11:45:20.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2300000000000001e-06 2024-09-16T11:45:20.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513315+0000", 2024-09-16T11:45:20.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039660590000000001 2024-09-16T11:45:20.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513328+0000", 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.308e-05 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513334+0000", 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513385+0000", 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1347000000000002e-05 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.513407+0000", 2024-09-16T11:45:20.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1852999999999999e-05 2024-09-16T11:45:20.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516882+0000", 2024-09-16T11:45:20.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034749690000000001 2024-09-16T11:45:20.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:00.516933+0000", 2024-09-16T11:45:20.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0952999999999998e-05 2024-09-16T11:45:20.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.105 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.105 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:28959 4.8 e53/28 4:15bee3c5:::rbd_data.11c1ce740b46.0000000000000001:head v 53'408, mlcod=53'406)", 2024-09-16T11:45:20.105 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:45:20.105 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 79.009829624000005, 2024-09-16T11:45:20.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0088740420000000004, 2024-09-16T11:45:20.105 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.105 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.105 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:45:20.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031497+0000", 2024-09-16T11:45:20.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.031498+0000", 2024-09-16T11:45:20.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.652e-06 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033515+0000", 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020165930000000001 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033539+0000", 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4295999999999999e-05 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033548+0000", 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1239999999999994e-06 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033610+0000", 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1371999999999994e-05 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.033637+0000", 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7098999999999999e-05 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040341+0000", 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0067045819999999997 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.040371+0000", 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9323999999999999e-05 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32303 4.8 e53/28 4:1436ac82:::rbd_data.11c1cab081b3.0000000000000022:head v 53'417, mlcod=53'415)", 2024-09-16T11:45:20.109 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:20.110 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.539403649999997, 2024-09-16T11:45:20.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0085565160000000001, 2024-09-16T11:45:20.110 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.110 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.110 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:20.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:20.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501923+0000", 2024-09-16T11:45:20.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3099999999999996e-07 2024-09-16T11:45:20.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501930+0000", 2024-09-16T11:45:20.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7649999999999996e-06 2024-09-16T11:45:20.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501946+0000", 2024-09-16T11:45:20.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5301000000000001e-05 2024-09-16T11:45:20.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501952+0000", 2024-09-16T11:45:20.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:45:20.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502100+0000", 2024-09-16T11:45:20.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014847600000000001 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502120+0000", 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9303000000000001e-05 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510436+0000", 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0083166420000000008 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510479+0000", 2024-09-16T11:45:20.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2979000000000003e-05 2024-09-16T11:45:20.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.114 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.114 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32306 4.1e e53/28 4:7a47e087:::rbd_data.11c1cab081b3.0000000000000025:head v 53'124, mlcod=53'122)", 2024-09-16T11:45:20.114 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:20.114 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.539337656000001, 2024-09-16T11:45:20.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0084990900000000008, 2024-09-16T11:45:20.114 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.114 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.114 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501989+0000", 2024-09-16T11:45:20.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.501994+0000", 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3239999999999996e-06 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502002+0000", 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3839999999999997e-06 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502007+0000", 2024-09-16T11:45:20.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6290000000000001e-06 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502454+0000", 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00044736800000000002 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502471+0000", 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6648999999999999e-05 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510455+0000", 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0079841909999999999 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510488+0000", 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2948999999999999e-05 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:32307 4.8 e53/28 4:11c1c37d:::rbd_data.11c1cab081b3.0000000000000026:head v 53'418, mlcod=53'415)", 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:20.118 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.538582321000007, 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00776212, 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502744+0000", 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502750+0000", 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502760+0000", 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:45:20.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502767+0000", 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4219999999999997e-06 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502802+0000", 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5089999999999998e-05 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.502816+0000", 2024-09-16T11:45:20.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4092e-05 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510483+0000", 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00766704 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:01.510506+0000", 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3195999999999999e-05 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33583 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'416, mlcod=53'411)", 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.830623911999993, 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078132459999999994, 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210702+0000", 2024-09-16T11:45:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210703+0000", 2024-09-16T11:45:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:45:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210706+0000", 2024-09-16T11:45:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1659999999999998e-06 2024-09-16T11:45:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210712+0000", 2024-09-16T11:45:20.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3890000000000002e-06 2024-09-16T11:45:20.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210715+0000", 2024-09-16T11:45:20.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:45:20.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210736+0000", 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0293999999999999e-05 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.210746+0000", 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0178e-05 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218477+0000", 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007731501 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.218516+0000", 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8148000000000001e-05 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33590 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'419, mlcod=53'415)", 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.819565771000001, 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0082499879999999998, 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.127 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.128 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:45:20.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221760+0000", 2024-09-16T11:45:20.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221761+0000", 2024-09-16T11:45:20.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-16T11:45:20.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221765+0000", 2024-09-16T11:45:20.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1590000000000003e-06 2024-09-16T11:45:20.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221774+0000", 2024-09-16T11:45:20.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7490000000000004e-06 2024-09-16T11:45:20.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221778+0000", 2024-09-16T11:45:20.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0419999999999996e-06 2024-09-16T11:45:20.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221800+0000", 2024-09-16T11:45:20.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1926999999999999e-05 2024-09-16T11:45:20.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.221810+0000", 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0079e-05 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.229980+0000", 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0081698469999999992 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230010+0000", 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0746e-05 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.132 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:33591 4.11 e53/28 4:89ad98cf:::rbd_object_map.11c1cab081b3:head v 53'420, mlcod=53'415)", 2024-09-16T11:45:20.132 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:20.132 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.819004286999998, 2024-09-16T11:45:20.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007702576, 2024-09-16T11:45:20.132 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.132 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.132 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:20.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:20.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222322+0000", 2024-09-16T11:45:20.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:45:20.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222326+0000", 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222337+0000", 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1008e-05 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222342+0000", 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0050000000000004e-06 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222366+0000", 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3696000000000001e-05 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.222385+0000", 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230012+0000", 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0076270940000000001 2024-09-16T11:45:20.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:02.230025+0000", 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2112e-05 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:55282 4.1e e73/28 4:785299e2:::rbd_data.11c11a49c2da.0000000000000001:head v 73'324, mlcod=73'322)", 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.802307767999999, 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0071431890000000003, 2024-09-16T11:45:20.136 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.137 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.137 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:45:20.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239018+0000", 2024-09-16T11:45:20.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.239020+0000", 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.049e-06 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244238+0000", 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052183489999999997 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244252+0000", 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4246999999999999e-05 2024-09-16T11:45:20.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244275+0000", 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3093999999999999e-05 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244314+0000", 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8495000000000003e-05 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.244330+0000", 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5917e-05 2024-09-16T11:45:20.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246121+0000", 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017909239999999999 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:24.246162+0000", 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1114e-05 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.140 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4545.0:61651 4.11 e83/28 4:8f4a51c5:::rbd_data.11c175f8cab7.0000000000000014:head v 83'694, mlcod=81'692)", 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.427888221000003, 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.009466482, 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613438+0000", 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.613439+0000", 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8400000000000003e-07 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618867+0000", 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0054276059999999998 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618883+0000", 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6265000000000001e-05 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618890+0000", 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2980000000000003e-06 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618950+0000", 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0025999999999998e-05 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.618971+0000", 2024-09-16T11:45:20.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0458999999999998e-05 2024-09-16T11:45:20.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622869+0000", 2024-09-16T11:45:20.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038983450000000001 2024-09-16T11:45:20.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:44:34.622905+0000", 2024-09-16T11:45:20.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599000000000003e-05 2024-09-16T11:45:20.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.144 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4545.0:95694 4.17 4:ec7bbff8:::rbd_data.11c18a2497a7.0000000000000008:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-16T11:45:13.626080+0000", 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.4152459539999995, 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0077344190000000002, 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4545", 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.70:0/3961522498", 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 95694 2024-09-16T11:45:20.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.146 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-16T11:45:20.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-16T11:45:20.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626080+0000", 2024-09-16T11:45:20.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-16T11:45:20.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626080+0000", 2024-09-16T11:45:20.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-16T11:45:20.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626083+0000", 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3180000000000001e-06 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626089+0000", 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626105+0000", 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5903999999999999e-05 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-16T11:45:20.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626112+0000", 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4649999999999997e-06 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626162+0000", 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0238999999999998e-05 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.626167+0000", 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8019999999999999e-06 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.628097+0000", 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019296719999999999 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.628165+0000", 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8313999999999997e-05 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.633698+0000", 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0055331410000000001 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.633742+0000", 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3408999999999997e-05 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-16T11:45:13.633815+0000", 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2935999999999994e-05 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-16T11:45:20.150 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-16T11:45:20.673 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:45:20.920 INFO:teuthology.orchestra.run.smithi070.stdout:get_pg_num_history pg_num_history pg_num_history(e112 pg_nums {1={14=1},2={16=8},3={20=4},4={28=32}} deleted_pools ) 2024-09-16T11:45:20.921 INFO:teuthology.orchestra.run.smithi070.stdout:Importing pgid 4.1b 2024-09-16T11:45:20.928 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:d857754c:::rbd_data.11c1b86c78bf.000000000000021d:head# 2024-09-16T11:45:20.928 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 13=[]:{} 2024-09-16T11:45:20.929 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:d86120ff:::rbd_data.11c1cbbf29d.0000000000000026:head# 2024-09-16T11:45:20.930 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 19=[]:{} 2024-09-16T11:45:20.930 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:d867928f:::rbd_data.11c1b86c78bf.000000000000023f:head# 2024-09-16T11:45:20.930 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 13=[]:{} 2024-09-16T11:45:20.930 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:d87206d0:::rbd_data.11c17a9d794a.0000000000000173:head# 2024-09-16T11:45:20.930 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.930 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:d89ddbb6:::rbd_data.11c17a9d794a.0000000000000166:head# 2024-09-16T11:45:20.930 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.930 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:d929272c:::rbd_data.11c17a9d794a.0000000000000150:head# 2024-09-16T11:45:20.930 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.931 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:d982b7e1:::rbd_data.11c17a9d794a.00000000000000c2:head# 2024-09-16T11:45:20.931 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.931 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:d9a074e9:::rbd_data.11c17a9d794a.0000000000000153:head# 2024-09-16T11:45:20.931 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.931 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:d9ffc0e1:::rbd_data.11c1ac01afc0.000000000000003f:head# 2024-09-16T11:45:20.931 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 0=[]:{} 2024-09-16T11:45:20.934 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:da3658b0:::rbd_data.11c1b86c78bf.0000000000000180:head# 2024-09-16T11:45:20.934 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 13=[]:{} 2024-09-16T11:45:20.934 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:da4bcbba:::rbd_data.11c130c94979.000000000000000d:6# 2024-09-16T11:45:20.935 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:da4bcbba:::rbd_data.11c130c94979.000000000000000d:head# 2024-09-16T11:45:20.935 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 6=[]:{6=[6]} 2024-09-16T11:45:20.935 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:da8d6b0b:::rbd_data.11c1b86c78bf.000000000000023e:head# 2024-09-16T11:45:20.935 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 13=[]:{} 2024-09-16T11:45:20.935 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:da9fad7f:::rbd_data.11c17a9d794a.0000000000000136:head# 2024-09-16T11:45:20.935 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.936 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:db162b91:::rbd_data.11c11284e6d8.0000000000000004:head# 2024-09-16T11:45:20.941 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 0=[]:{} 2024-09-16T11:45:20.947 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:db62f8f9:::rbd_data.11c1b86c78bf.00000000000001b6:head# 2024-09-16T11:45:20.947 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 13=[]:{} 2024-09-16T11:45:20.948 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dbc18110:::rbd_data.11c11778572.000000000000000e:head# 2024-09-16T11:45:20.948 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 12=[]:{} 2024-09-16T11:45:20.948 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dbead158:::rbd_data.11c17a9d794a.0000000000000109:head# 2024-09-16T11:45:20.948 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.948 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dc9de789:::rbd_data.11c17a9d794a.0000000000000122:head# 2024-09-16T11:45:20.948 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.949 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dcbdc6eb:::rbd_data.11c17a9d794a.00000000000000f9:head# 2024-09-16T11:45:20.949 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.949 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dcc5f862:::rbd_data.11c130c94979.000000000000000b:6# 2024-09-16T11:45:20.949 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dcc5f862:::rbd_data.11c130c94979.000000000000000b:head# 2024-09-16T11:45:20.950 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 6=[]:{6=[6]} 2024-09-16T11:45:20.950 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dd0bf4c8:::rbd_data.11c17a9d794a.0000000000000172:head# 2024-09-16T11:45:20.950 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.951 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dd7391d6:::rbd_data.11c17a9d794a.000000000000004c:head# 2024-09-16T11:45:20.951 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.951 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:ddec7dc0:::rbd_data.11c17a9d794a.000000000000015a:head# 2024-09-16T11:45:20.951 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.951 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:de2906cb:::rbd_data.11c17a9d794a.00000000000000e3:head# 2024-09-16T11:45:20.951 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.951 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:de2ddba8:::rbd_data.11c1b86c78bf.000000000000024b:head# 2024-09-16T11:45:20.952 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 13=[]:{} 2024-09-16T11:45:20.952 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:de51de82:::rbd_data.11c17a9d794a.00000000000000dd:head# 2024-09-16T11:45:20.952 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.952 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:de84200e:::rbd_data.11c1ac01afc0.0000000000000004:head# 2024-09-16T11:45:20.952 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 0=[]:{} 2024-09-16T11:45:20.953 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:de86f447:::rbd_data.11c11778572.0000000000000007:head# 2024-09-16T11:45:20.953 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 12=[]:{} 2024-09-16T11:45:20.953 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:df421a3f:::rbd_header.11c1e24d7fee:head# 2024-09-16T11:45:20.953 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 0=[]:{} 2024-09-16T11:45:20.953 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:df63fd3b:::rbd_data.11c1b86c78bf.000000000000022d:head# 2024-09-16T11:45:20.953 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 13=[]:{} 2024-09-16T11:45:20.954 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dfadecd0:::rbd_data.11c1b86c78bf.0000000000000228:head# 2024-09-16T11:45:20.954 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 13=[]:{} 2024-09-16T11:45:20.954 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dfca7b45:::rbd_data.11c17a9d794a.000000000000017d:head# 2024-09-16T11:45:20.954 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 15=[]:{} 2024-09-16T11:45:20.954 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:5# 2024-09-16T11:45:20.955 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dfcd994b:::rbd_data.11c17b8dc4e5.0000000000000000:head# 2024-09-16T11:45:20.955 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 5=[]:{5=[5]} 2024-09-16T11:45:20.957 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dfdc27f6:::rbd_data.11c1b86c78bf.000000000000014e:head# 2024-09-16T11:45:20.957 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 13=[]:{} 2024-09-16T11:45:20.958 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dfe5f588:::rbd_data.11c12816a66b.000000000000000d:head# 2024-09-16T11:45:20.958 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 9=[]:{} 2024-09-16T11:45:20.960 INFO:teuthology.orchestra.run.smithi070.stdout:Write #4:dff4be41:::rbd_data.11c12e336783.0000000000000002:head# 2024-09-16T11:45:20.961 INFO:teuthology.orchestra.run.smithi070.stdout:snapset 0=[]:{} 2024-09-16T11:45:20.961 INFO:teuthology.orchestra.run.smithi070.stdout:write_pg epoch 112 info 4.1b( v 108'346 (0'0,108'346] local-lis/les=107/108 n=39 ec=28/28 lis/c=107/107 les/c/f=108/108/0 sis=107) 2024-09-16T11:45:20.964 INFO:teuthology.orchestra.run.smithi070.stdout:Import successful 2024-09-16T11:45:21.479 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /var/log/ceph/exp.4.1b.3 2024-09-16T11:45:21.509 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:45:21.600 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T11:45:21.613 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:45:21.825 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:45:21.826 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":128,"fsid":"3420bcad-625f-4e15-aaff-fb4236322168","created":"2024-09-16T11:43:01.654626+0000","modified":"2024-09-16T11:45:21.591462+0000","last_up_change":"2024-09-16T11:45:05.523616+0000","last_in_change":"2024-09-16T11:43:04.526236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:43:11.406582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:43:13.125709+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"23","last_force_op_resend_prenautilus":"23","last_force_op_resend_preluminous":"23","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T11:43:21.227888+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"23","last_force_op_resend_prenautilus":"23","last_force_op_resend_preluminous":"23","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T11:43:30.105533+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":83,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"91f052e4-4c8f-4484-92ce-bd1a7f8f3efb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":33355},{"type":"v1","addr":"172.21.15.70:6801","nonce":33355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":33355},{"type":"v1","addr":"172.21.15.70:6803","nonce":33355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":33355},{"type":"v1","addr":"172.21.15.70:6807","nonce":33355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":33355},{"type":"v1","addr":"172.21.15.70:6805","nonce":33355}]},"public_addr":"172.21.15.70:6801/33355","cluster_addr":"172.21.15.70:6803/33355","heartbeat_back_addr":"172.21.15.70:6807/33355","heartbeat_front_addr":"172.21.15.70:6805/33355","state":["exists","up"]},{"osd":1,"uuid":"568b1280-7365-4094-afe8-b0b5e766de6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":33359},{"type":"v1","addr":"172.21.15.70:6817","nonce":33359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":33359},{"type":"v1","addr":"172.21.15.70:6819","nonce":33359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":33359},{"type":"v1","addr":"172.21.15.70:6823","nonce":33359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":33359},{"type":"v1","addr":"172.21.15.70:6821","nonce":33359}]},"public_addr":"172.21.15.70:6817/33359","cluster_addr":"172.21.15.70:6819/33359","heartbeat_back_addr":"172.21.15.70:6823/33359","heartbeat_front_addr":"172.21.15.70:6821/33359","state":["exists","up"]},{"osd":2,"uuid":"328adfa3-c1b4-4745-9759-7327c8cbd499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":33356},{"type":"v1","addr":"172.21.15.70:6809","nonce":33356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":33356},{"type":"v1","addr":"172.21.15.70:6811","nonce":33356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":33356},{"type":"v1","addr":"172.21.15.70:6815","nonce":33356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":33356},{"type":"v1","addr":"172.21.15.70:6813","nonce":33356}]},"public_addr":"172.21.15.70:6809/33356","cluster_addr":"172.21.15.70:6811/33356","heartbeat_back_addr":"172.21.15.70:6815/33356","heartbeat_front_addr":"172.21.15.70:6813/33356","state":["exists","up"]},{"osd":3,"uuid":"6df66ae7-e835-4d8f-bd27-243d7a9705b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":113,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":33384},{"type":"v1","addr":"172.21.15.70:6825","nonce":33384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":33384},{"type":"v1","addr":"172.21.15.70:6827","nonce":33384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":33384},{"type":"v1","addr":"172.21.15.70:6831","nonce":33384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":33384},{"type":"v1","addr":"172.21.15.70:6829","nonce":33384}]},"public_addr":"172.21.15.70:6825/33384","cluster_addr":"172.21.15.70:6827/33384","heartbeat_back_addr":"172.21.15.70:6831/33384","heartbeat_front_addr":"172.21.15.70:6829/33384","state":["exists"]},{"osd":4,"uuid":"29f52d62-d4c3-49f9-992f-23817f5622d2","up":1,"in":1,"weight":1,"primary_affinity":0.4756011962890625,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":33331},{"type":"v1","addr":"172.21.15.192:6803","nonce":33331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":33331},{"type":"v1","addr":"172.21.15.192:6807","nonce":33331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6813","nonce":33331},{"type":"v1","addr":"172.21.15.192:6815","nonce":33331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":33331},{"type":"v1","addr":"172.21.15.192:6811","nonce":33331}]},"public_addr":"172.21.15.192:6803/33331","cluster_addr":"172.21.15.192:6807/33331","heartbeat_back_addr":"172.21.15.192:6815/33331","heartbeat_front_addr":"172.21.15.192:6811/33331","state":["exists","up"]},{"osd":5,"uuid":"0c1a275a-3c75-4e0b-bd35-d34b64d99c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":33333},{"type":"v1","addr":"172.21.15.192:6817","nonce":33333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":33333},{"type":"v1","addr":"172.21.15.192:6819","nonce":33333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":33333},{"type":"v1","addr":"172.21.15.192:6823","nonce":33333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":33333},{"type":"v1","addr":"172.21.15.192:6821","nonce":33333}]},"public_addr":"172.21.15.192:6817/33333","cluster_addr":"172.21.15.192:6819/33333","heartbeat_back_addr":"172.21.15.192:6823/33333","heartbeat_front_addr":"172.21.15.192:6821/33333","state":["exists","up"]},{"osd":6,"uuid":"d79f0070-ce58-48e7-a56a-2762ec76a271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6801","nonce":33332},{"type":"v1","addr":"172.21.15.192:6802","nonce":33332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":33332},{"type":"v1","addr":"172.21.15.192:6806","nonce":33332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":33332},{"type":"v1","addr":"172.21.15.192:6814","nonce":33332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":33332},{"type":"v1","addr":"172.21.15.192:6810","nonce":33332}]},"public_addr":"172.21.15.192:6802/33332","cluster_addr":"172.21.15.192:6806/33332","heartbeat_back_addr":"172.21.15.192:6814/33332","heartbeat_front_addr":"172.21.15.192:6810/33332","state":["exists","up"]},{"osd":7,"uuid":"537fa5df-3d0a-4dd4-8155-20ee94e41c82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":33362},{"type":"v1","addr":"172.21.15.192:6825","nonce":33362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":33362},{"type":"v1","addr":"172.21.15.192:6827","nonce":33362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":33362},{"type":"v1","addr":"172.21.15.192:6831","nonce":33362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":33362},{"type":"v1","addr":"172.21.15.192:6829","nonce":33362}]},"public_addr":"172.21.15.192:6825/33362","cluster_addr":"172.21.15.192:6827/33362","heartbeat_back_addr":"172.21.15.192:6831/33362","heartbeat_front_addr":"172.21.15.192:6829/33362","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.581514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.674271+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.545402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.733565+0000","dead_epoch":113},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.623479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.730066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.556650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.702368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":5,"to":1}]},{"pgid":"4.14","mappings":[{"from":1,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":4,"snaps":[{"begin":4,"length":13}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:45:21.838 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T11:43:11.406582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T11:43:13.125709+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '23', 'last_force_op_resend_prenautilus': '23', 'last_force_op_resend_preluminous': '23', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-09-16T11:43:21.227888+0000', 'flags': 8201, 'flags_names': 'hashpspool,incomplete_clones,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '23', 'last_force_op_resend_prenautilus': '23', 'last_force_op_resend_preluminous': '23', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T11:43:30.105533+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 83, 'snap_epoch': 127, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-09-16T11:45:21.839 DEBUG:teuthology.orchestra.run.smithi070:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op apply-layout-settings --pool .mgr 2024-09-16T11:45:22.605 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T11:45:22.937 INFO:teuthology.orchestra.run.smithi070.stderr:Nothing to do for non-filestore backend 2024-09-16T11:45:23.455 DEBUG:teuthology.orchestra.run.smithi070:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op apply-layout-settings --pool rbd 2024-09-16T11:45:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:Nothing to do for non-filestore backend 2024-09-16T11:45:25.060 DEBUG:teuthology.orchestra.run.smithi070:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op apply-layout-settings --pool cache 2024-09-16T11:45:26.155 INFO:teuthology.orchestra.run.smithi070.stderr:Nothing to do for non-filestore backend 2024-09-16T11:45:26.672 DEBUG:teuthology.orchestra.run.smithi070:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op apply-layout-settings --pool pool_client.0 2024-09-16T11:45:27.757 INFO:teuthology.orchestra.run.smithi070.stderr:Nothing to do for non-filestore backend 2024-09-16T11:45:33.279 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-16T11:45:33.279 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-16T11:45:33.280 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T11:45:33.280 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T11:45:33.283 INFO:tasks.ceph.osd.3:Started 2024-09-16T11:45:33.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:45:33.389 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T11:45:33.389 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T11:45:33.389 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T11:45:33.389 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T11:45:33.389 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T11:45:33.389 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T11:45:33.389 INFO:teuthology.orchestra.run.smithi070.stderr:config diff 2024-09-16T11:45:33.389 INFO:teuthology.orchestra.run.smithi070.stderr:config diff get 2024-09-16T11:45:33.389 INFO:teuthology.orchestra.run.smithi070.stderr:config get 2024-09-16T11:45:33.389 INFO:teuthology.orchestra.run.smithi070.stderr:config help [] 2024-09-16T11:45:33.389 INFO:teuthology.orchestra.run.smithi070.stderr:config set ... 2024-09-16T11:45:33.389 INFO:teuthology.orchestra.run.smithi070.stderr:admin_socket: invalid command 2024-09-16T11:45:33.391 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T11:45:33.391 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-16T11:45:33.876 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:45:33.873+0000 7eff5728b440 -1 Falling back to public interface 2024-09-16T11:45:37.395 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:45:37.393+0000 7eff5728b440 -1 osd.3 112 log_to_monitors true 2024-09-16T11:45:37.646 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-16T11:45:37.644+0000 7eff4b284640 -1 osd.3 112 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:45:38.393 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:45:38.506 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:45:38.506 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T11:45:38.506 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T11:45:38.507 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:45:38.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-16T11:45:38.622 INFO:teuthology.orchestra.run.smithi070.stdout:mon.a: {} 2024-09-16T11:45:38.622 INFO:teuthology.orchestra.run.smithi070.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T11:45:38.636 INFO:teuthology.orchestra.run.smithi070.stdout:mon.b: {} 2024-09-16T11:45:38.636 INFO:teuthology.orchestra.run.smithi070.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T11:45:38.647 INFO:teuthology.orchestra.run.smithi070.stdout:mon.c: {} 2024-09-16T11:45:38.647 INFO:teuthology.orchestra.run.smithi070.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T11:45:38.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T11:45:39.633 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T11:45:39.660 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-16T11:45:40.644 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-16T11:45:40.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T11:45:41.651 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T11:45:41.667 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-16T11:45:42.653 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-16T11:45:42.668 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T11:45:43.665 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T11:45:43.678 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-16T11:45:44.663 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-16T11:45:44.678 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T11:45:45.668 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T11:45:45.683 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-16T11:45:46.457 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-16T11:45:46.472 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T11:45:47.459 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T11:45:47.478 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-16T11:45:48.680 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-16T11:45:48.697 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T11:45:49.686 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T11:45:49.702 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-16T11:45:50.690 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-16T11:45:50.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T11:45:51.695 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T11:45:51.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-16T11:45:52.700 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-16T11:45:52.716 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T11:45:53.705 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T11:45:53.721 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-16T11:45:54.711 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-16T11:45:54.728 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-16T11:45:54.728 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-16T11:45:54.728 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-16T11:45:54.728 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-16T11:45:54.728 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T11:45:54.729 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:45:54.996 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:45:54.997 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":147,"fsid":"3420bcad-625f-4e15-aaff-fb4236322168","created":"2024-09-16T11:43:01.654626+0000","modified":"2024-09-16T11:45:54.705133+0000","last_up_change":"2024-09-16T11:45:38.626538+0000","last_in_change":"2024-09-16T11:43:04.526236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:43:11.406582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:43:13.125709+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"23","last_force_op_resend_prenautilus":"23","last_force_op_resend_preluminous":"23","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T11:43:21.227888+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"23","last_force_op_resend_prenautilus":"23","last_force_op_resend_preluminous":"23","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T11:43:30.105533+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":83,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"91f052e4-4c8f-4484-92ce-bd1a7f8f3efb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":33355},{"type":"v1","addr":"172.21.15.70:6801","nonce":33355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":33355},{"type":"v1","addr":"172.21.15.70:6803","nonce":33355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":33355},{"type":"v1","addr":"172.21.15.70:6807","nonce":33355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":33355},{"type":"v1","addr":"172.21.15.70:6805","nonce":33355}]},"public_addr":"172.21.15.70:6801/33355","cluster_addr":"172.21.15.70:6803/33355","heartbeat_back_addr":"172.21.15.70:6807/33355","heartbeat_front_addr":"172.21.15.70:6805/33355","state":["exists","up"]},{"osd":1,"uuid":"568b1280-7365-4094-afe8-b0b5e766de6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":33359},{"type":"v1","addr":"172.21.15.70:6817","nonce":33359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":33359},{"type":"v1","addr":"172.21.15.70:6819","nonce":33359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":33359},{"type":"v1","addr":"172.21.15.70:6823","nonce":33359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":33359},{"type":"v1","addr":"172.21.15.70:6821","nonce":33359}]},"public_addr":"172.21.15.70:6817/33359","cluster_addr":"172.21.15.70:6819/33359","heartbeat_back_addr":"172.21.15.70:6823/33359","heartbeat_front_addr":"172.21.15.70:6821/33359","state":["exists","up"]},{"osd":2,"uuid":"328adfa3-c1b4-4745-9759-7327c8cbd499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":33356},{"type":"v1","addr":"172.21.15.70:6809","nonce":33356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":33356},{"type":"v1","addr":"172.21.15.70:6811","nonce":33356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":33356},{"type":"v1","addr":"172.21.15.70:6815","nonce":33356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":33356},{"type":"v1","addr":"172.21.15.70:6813","nonce":33356}]},"public_addr":"172.21.15.70:6809/33356","cluster_addr":"172.21.15.70:6811/33356","heartbeat_back_addr":"172.21.15.70:6815/33356","heartbeat_front_addr":"172.21.15.70:6813/33356","state":["exists","up"]},{"osd":3,"uuid":"6df66ae7-e835-4d8f-bd27-243d7a9705b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":112,"up_from":131,"up_thru":131,"down_at":113,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":58254},{"type":"v1","addr":"172.21.15.70:6825","nonce":58254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":58254},{"type":"v1","addr":"172.21.15.70:6827","nonce":58254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":58254},{"type":"v1","addr":"172.21.15.70:6831","nonce":58254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":58254},{"type":"v1","addr":"172.21.15.70:6829","nonce":58254}]},"public_addr":"172.21.15.70:6825/58254","cluster_addr":"172.21.15.70:6827/58254","heartbeat_back_addr":"172.21.15.70:6831/58254","heartbeat_front_addr":"172.21.15.70:6829/58254","state":["exists","up"]},{"osd":4,"uuid":"29f52d62-d4c3-49f9-992f-23817f5622d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":33331},{"type":"v1","addr":"172.21.15.192:6803","nonce":33331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":33331},{"type":"v1","addr":"172.21.15.192:6807","nonce":33331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6813","nonce":33331},{"type":"v1","addr":"172.21.15.192:6815","nonce":33331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":33331},{"type":"v1","addr":"172.21.15.192:6811","nonce":33331}]},"public_addr":"172.21.15.192:6803/33331","cluster_addr":"172.21.15.192:6807/33331","heartbeat_back_addr":"172.21.15.192:6815/33331","heartbeat_front_addr":"172.21.15.192:6811/33331","state":["exists","up"]},{"osd":5,"uuid":"0c1a275a-3c75-4e0b-bd35-d34b64d99c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":33333},{"type":"v1","addr":"172.21.15.192:6817","nonce":33333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":33333},{"type":"v1","addr":"172.21.15.192:6819","nonce":33333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":33333},{"type":"v1","addr":"172.21.15.192:6823","nonce":33333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":33333},{"type":"v1","addr":"172.21.15.192:6821","nonce":33333}]},"public_addr":"172.21.15.192:6817/33333","cluster_addr":"172.21.15.192:6819/33333","heartbeat_back_addr":"172.21.15.192:6823/33333","heartbeat_front_addr":"172.21.15.192:6821/33333","state":["exists","up"]},{"osd":6,"uuid":"d79f0070-ce58-48e7-a56a-2762ec76a271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6801","nonce":33332},{"type":"v1","addr":"172.21.15.192:6802","nonce":33332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":33332},{"type":"v1","addr":"172.21.15.192:6806","nonce":33332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":33332},{"type":"v1","addr":"172.21.15.192:6814","nonce":33332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":33332},{"type":"v1","addr":"172.21.15.192:6810","nonce":33332}]},"public_addr":"172.21.15.192:6802/33332","cluster_addr":"172.21.15.192:6806/33332","heartbeat_back_addr":"172.21.15.192:6814/33332","heartbeat_front_addr":"172.21.15.192:6810/33332","state":["exists","up"]},{"osd":7,"uuid":"537fa5df-3d0a-4dd4-8155-20ee94e41c82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":33362},{"type":"v1","addr":"172.21.15.192:6825","nonce":33362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":33362},{"type":"v1","addr":"172.21.15.192:6827","nonce":33362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":33362},{"type":"v1","addr":"172.21.15.192:6831","nonce":33362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":33362},{"type":"v1","addr":"172.21.15.192:6829","nonce":33362}]},"public_addr":"172.21.15.192:6825/33362","cluster_addr":"172.21.15.192:6827/33362","heartbeat_back_addr":"172.21.15.192:6831/33362","heartbeat_front_addr":"172.21.15.192:6829/33362","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.581514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.674271+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.545402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.733565+0000","dead_epoch":113},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.623479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.730066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.556650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.702368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":5,"to":1}]},{"pgid":"4.14","mappings":[{"from":1,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:45:55.009 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T11:45:55.010 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:45:55.269 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:45:55.270 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":147,"fsid":"3420bcad-625f-4e15-aaff-fb4236322168","created":"2024-09-16T11:43:01.654626+0000","modified":"2024-09-16T11:45:54.705133+0000","last_up_change":"2024-09-16T11:45:38.626538+0000","last_in_change":"2024-09-16T11:43:04.526236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:43:11.406582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:43:13.125709+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"23","last_force_op_resend_prenautilus":"23","last_force_op_resend_preluminous":"23","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T11:43:21.227888+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"23","last_force_op_resend_prenautilus":"23","last_force_op_resend_preluminous":"23","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T11:43:30.105533+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":83,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"91f052e4-4c8f-4484-92ce-bd1a7f8f3efb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":33355},{"type":"v1","addr":"172.21.15.70:6801","nonce":33355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":33355},{"type":"v1","addr":"172.21.15.70:6803","nonce":33355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":33355},{"type":"v1","addr":"172.21.15.70:6807","nonce":33355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":33355},{"type":"v1","addr":"172.21.15.70:6805","nonce":33355}]},"public_addr":"172.21.15.70:6801/33355","cluster_addr":"172.21.15.70:6803/33355","heartbeat_back_addr":"172.21.15.70:6807/33355","heartbeat_front_addr":"172.21.15.70:6805/33355","state":["exists","up"]},{"osd":1,"uuid":"568b1280-7365-4094-afe8-b0b5e766de6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":33359},{"type":"v1","addr":"172.21.15.70:6817","nonce":33359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":33359},{"type":"v1","addr":"172.21.15.70:6819","nonce":33359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":33359},{"type":"v1","addr":"172.21.15.70:6823","nonce":33359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":33359},{"type":"v1","addr":"172.21.15.70:6821","nonce":33359}]},"public_addr":"172.21.15.70:6817/33359","cluster_addr":"172.21.15.70:6819/33359","heartbeat_back_addr":"172.21.15.70:6823/33359","heartbeat_front_addr":"172.21.15.70:6821/33359","state":["exists","up"]},{"osd":2,"uuid":"328adfa3-c1b4-4745-9759-7327c8cbd499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":33356},{"type":"v1","addr":"172.21.15.70:6809","nonce":33356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":33356},{"type":"v1","addr":"172.21.15.70:6811","nonce":33356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":33356},{"type":"v1","addr":"172.21.15.70:6815","nonce":33356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":33356},{"type":"v1","addr":"172.21.15.70:6813","nonce":33356}]},"public_addr":"172.21.15.70:6809/33356","cluster_addr":"172.21.15.70:6811/33356","heartbeat_back_addr":"172.21.15.70:6815/33356","heartbeat_front_addr":"172.21.15.70:6813/33356","state":["exists","up"]},{"osd":3,"uuid":"6df66ae7-e835-4d8f-bd27-243d7a9705b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":112,"up_from":131,"up_thru":131,"down_at":113,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":58254},{"type":"v1","addr":"172.21.15.70:6825","nonce":58254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":58254},{"type":"v1","addr":"172.21.15.70:6827","nonce":58254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":58254},{"type":"v1","addr":"172.21.15.70:6831","nonce":58254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":58254},{"type":"v1","addr":"172.21.15.70:6829","nonce":58254}]},"public_addr":"172.21.15.70:6825/58254","cluster_addr":"172.21.15.70:6827/58254","heartbeat_back_addr":"172.21.15.70:6831/58254","heartbeat_front_addr":"172.21.15.70:6829/58254","state":["exists","up"]},{"osd":4,"uuid":"29f52d62-d4c3-49f9-992f-23817f5622d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":33331},{"type":"v1","addr":"172.21.15.192:6803","nonce":33331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":33331},{"type":"v1","addr":"172.21.15.192:6807","nonce":33331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6813","nonce":33331},{"type":"v1","addr":"172.21.15.192:6815","nonce":33331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":33331},{"type":"v1","addr":"172.21.15.192:6811","nonce":33331}]},"public_addr":"172.21.15.192:6803/33331","cluster_addr":"172.21.15.192:6807/33331","heartbeat_back_addr":"172.21.15.192:6815/33331","heartbeat_front_addr":"172.21.15.192:6811/33331","state":["exists","up"]},{"osd":5,"uuid":"0c1a275a-3c75-4e0b-bd35-d34b64d99c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":33333},{"type":"v1","addr":"172.21.15.192:6817","nonce":33333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":33333},{"type":"v1","addr":"172.21.15.192:6819","nonce":33333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":33333},{"type":"v1","addr":"172.21.15.192:6823","nonce":33333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":33333},{"type":"v1","addr":"172.21.15.192:6821","nonce":33333}]},"public_addr":"172.21.15.192:6817/33333","cluster_addr":"172.21.15.192:6819/33333","heartbeat_back_addr":"172.21.15.192:6823/33333","heartbeat_front_addr":"172.21.15.192:6821/33333","state":["exists","up"]},{"osd":6,"uuid":"d79f0070-ce58-48e7-a56a-2762ec76a271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6801","nonce":33332},{"type":"v1","addr":"172.21.15.192:6802","nonce":33332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":33332},{"type":"v1","addr":"172.21.15.192:6806","nonce":33332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":33332},{"type":"v1","addr":"172.21.15.192:6814","nonce":33332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":33332},{"type":"v1","addr":"172.21.15.192:6810","nonce":33332}]},"public_addr":"172.21.15.192:6802/33332","cluster_addr":"172.21.15.192:6806/33332","heartbeat_back_addr":"172.21.15.192:6814/33332","heartbeat_front_addr":"172.21.15.192:6810/33332","state":["exists","up"]},{"osd":7,"uuid":"537fa5df-3d0a-4dd4-8155-20ee94e41c82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":33362},{"type":"v1","addr":"172.21.15.192:6825","nonce":33362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":33362},{"type":"v1","addr":"172.21.15.192:6827","nonce":33362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":33362},{"type":"v1","addr":"172.21.15.192:6831","nonce":33362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":33362},{"type":"v1","addr":"172.21.15.192:6829","nonce":33362}]},"public_addr":"172.21.15.192:6825/33362","cluster_addr":"172.21.15.192:6827/33362","heartbeat_back_addr":"172.21.15.192:6831/33362","heartbeat_front_addr":"172.21.15.192:6829/33362","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.581514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.674271+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.545402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.733565+0000","dead_epoch":113},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.623479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.730066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.556650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.702368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":5,"to":1}]},{"pgid":"4.14","mappings":[{"from":1,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:45:55.282 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T11:45:55.282 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T11:45:55.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T11:45:55.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T11:45:55.284 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T11:45:55.284 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T11:45:55.284 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T11:45:55.285 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T11:45:55.499 INFO:teuthology.orchestra.run.smithi070.stdout:60129542179 2024-09-16T11:45:55.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T11:45:55.504 INFO:teuthology.orchestra.run.smithi070.stdout:55834574883 2024-09-16T11:45:55.505 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T11:45:55.505 INFO:teuthology.orchestra.run.smithi070.stdout:562640715781 2024-09-16T11:45:55.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T11:45:55.509 INFO:teuthology.orchestra.run.smithi070.stdout:55834574883 2024-09-16T11:45:55.509 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T11:45:55.511 INFO:teuthology.orchestra.run.smithi070.stdout:60129542180 2024-09-16T11:45:55.511 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T11:45:55.512 INFO:teuthology.orchestra.run.smithi070.stdout:55834574883 2024-09-16T11:45:55.512 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T11:45:55.514 INFO:teuthology.orchestra.run.smithi070.stdout:60129542180 2024-09-16T11:45:55.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T11:45:55.715 INFO:teuthology.orchestra.run.smithi070.stdout:55834574883 2024-09-16T11:45:55.715 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T11:45:55.980 INFO:teuthology.orchestra.run.smithi070.stdout:562640715780 2024-09-16T11:45:55.994 INFO:teuthology.orchestra.run.smithi070.stdout:55834574882 2024-09-16T11:45:55.996 INFO:teuthology.orchestra.run.smithi070.stdout:60129542178 2024-09-16T11:45:56.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 562640715781 got 562640715780 for osd.3 2024-09-16T11:45:56.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.6 2024-09-16T11:45:56.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542179 got 60129542178 for osd.5 2024-09-16T11:45:56.045 INFO:teuthology.orchestra.run.smithi070.stdout:55834574882 2024-09-16T11:45:56.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.2 2024-09-16T11:45:56.066 INFO:teuthology.orchestra.run.smithi070.stdout:60129542179 2024-09-16T11:45:56.078 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542180 got 60129542179 for osd.7 2024-09-16T11:45:56.083 INFO:teuthology.orchestra.run.smithi070.stdout:60129542179 2024-09-16T11:45:56.093 INFO:teuthology.orchestra.run.smithi070.stdout:55834574882 2024-09-16T11:45:56.095 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542180 got 60129542179 for osd.1 2024-09-16T11:45:56.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.4 2024-09-16T11:45:56.130 INFO:teuthology.orchestra.run.smithi070.stdout:55834574882 2024-09-16T11:45:56.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.0 2024-09-16T11:45:57.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T11:45:57.017 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T11:45:57.017 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T11:45:57.059 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T11:45:57.080 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T11:45:57.096 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T11:45:57.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T11:45:57.143 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T11:45:57.451 INFO:teuthology.orchestra.run.smithi070.stdout:562640715780 2024-09-16T11:45:57.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 562640715781 got 562640715780 for osd.3 2024-09-16T11:45:57.530 INFO:teuthology.orchestra.run.smithi070.stdout:60129542178 2024-09-16T11:45:57.532 INFO:teuthology.orchestra.run.smithi070.stdout:55834574882 2024-09-16T11:45:57.544 INFO:teuthology.orchestra.run.smithi070.stdout:60129542179 2024-09-16T11:45:57.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.6 2024-09-16T11:45:57.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542179 got 60129542178 for osd.5 2024-09-16T11:45:57.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542180 got 60129542179 for osd.7 2024-09-16T11:45:57.575 INFO:teuthology.orchestra.run.smithi070.stdout:55834574882 2024-09-16T11:45:57.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.2 2024-09-16T11:45:57.606 INFO:teuthology.orchestra.run.smithi070.stdout:60129542179 2024-09-16T11:45:57.611 INFO:teuthology.orchestra.run.smithi070.stdout:55834574882 2024-09-16T11:45:57.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542180 got 60129542179 for osd.1 2024-09-16T11:45:57.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.4 2024-09-16T11:45:57.631 INFO:teuthology.orchestra.run.smithi070.stdout:55834574882 2024-09-16T11:45:57.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.0 2024-09-16T11:45:58.470 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T11:45:58.553 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T11:45:58.553 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T11:45:58.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T11:45:58.589 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T11:45:58.621 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T11:45:58.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T11:45:58.645 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T11:45:58.935 INFO:teuthology.orchestra.run.smithi070.stdout:562640715781 2024-09-16T11:45:58.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 562640715781 got 562640715781 for osd.3 2024-09-16T11:45:58.957 DEBUG:teuthology.parallel:result is None 2024-09-16T11:45:59.031 INFO:teuthology.orchestra.run.smithi070.stdout:60129542180 2024-09-16T11:45:59.043 INFO:teuthology.orchestra.run.smithi070.stdout:60129542179 2024-09-16T11:45:59.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542180 got 60129542180 for osd.7 2024-09-16T11:45:59.046 DEBUG:teuthology.parallel:result is None 2024-09-16T11:45:59.056 INFO:teuthology.orchestra.run.smithi070.stdout:55834574884 2024-09-16T11:45:59.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542179 got 60129542179 for osd.5 2024-09-16T11:45:59.058 DEBUG:teuthology.parallel:result is None 2024-09-16T11:45:59.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574884 for osd.6 2024-09-16T11:45:59.071 DEBUG:teuthology.parallel:result is None 2024-09-16T11:45:59.102 INFO:teuthology.orchestra.run.smithi070.stdout:55834574884 2024-09-16T11:45:59.113 INFO:teuthology.orchestra.run.smithi070.stdout:55834574884 2024-09-16T11:45:59.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574884 for osd.2 2024-09-16T11:45:59.119 DEBUG:teuthology.parallel:result is None 2024-09-16T11:45:59.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574884 for osd.4 2024-09-16T11:45:59.126 DEBUG:teuthology.parallel:result is None 2024-09-16T11:45:59.136 INFO:teuthology.orchestra.run.smithi070.stdout:55834574884 2024-09-16T11:45:59.146 INFO:teuthology.orchestra.run.smithi070.stdout:60129542180 2024-09-16T11:45:59.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574884 for osd.0 2024-09-16T11:45:59.149 DEBUG:teuthology.parallel:result is None 2024-09-16T11:45:59.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542180 got 60129542180 for osd.1 2024-09-16T11:45:59.159 DEBUG:teuthology.parallel:result is None 2024-09-16T11:45:59.159 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T11:45:59.159 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:45:59.462 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:45:59.462 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:45:59.473 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-16T11:45:59.402327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590574,"num_objects":741,"num_object_clones":0,"num_object_copies":1482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":731,"num_read":135541,"num_read_kb":642951,"num_write":18165,"num_write_kb":292238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16567,"ondisk_log_size":16567,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445748,"kb_used_data":13884,"kb_used_omap":54,"kb_used_meta":2431689,"kb_avail":752528972,"statfs":{"total":773094113280,"available":770589667328,"internally_reserved":0,"allocated":14217216,"data_stored":8312404,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56156,"internal_metadata":2490049700},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.705432"},"pg_stats":[{"pgid":"4.1f","version":"127'327","reported_seq":6237,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725885+0000","last_change":"2024-09-16T11:45:19.597844+0000","last_active":"2024-09-16T11:45:54.725885+0000","last_peered":"2024-09-16T11:45:54.725885+0000","last_clean":"2024-09-16T11:45:54.725885+0000","last_became_active":"2024-09-16T11:43:31.779428+0000","last_became_peered":"2024-09-16T11:43:31.779428+0000","last_unstale":"2024-09-16T11:45:54.725885+0000","last_undegraded":"2024-09-16T11:45:54.725885+0000","last_fullsized":"2024-09-16T11:45:54.725885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:30.437698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034041099999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6762,"num_read_kb":29123,"num_write":304,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"125'449","reported_seq":5346,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566266+0000","last_change":"2024-09-16T11:45:19.592718+0000","last_active":"2024-09-16T11:45:54.566266+0000","last_peered":"2024-09-16T11:45:54.566266+0000","last_clean":"2024-09-16T11:45:54.566266+0000","last_became_active":"2024-09-16T11:43:31.889706+0000","last_became_peered":"2024-09-16T11:43:31.889706+0000","last_unstale":"2024-09-16T11:45:54.566266+0000","last_undegraded":"2024-09-16T11:45:54.566266+0000","last_fullsized":"2024-09-16T11:45:54.566266+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:25.242116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000654113,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5672,"num_read_kb":36055,"num_write":486,"num_write_kb":9211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"132'213","reported_seq":1539,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725493+0000","last_change":"2024-09-16T11:45:39.661471+0000","last_active":"2024-09-16T11:45:54.725493+0000","last_peered":"2024-09-16T11:45:54.725493+0000","last_clean":"2024-09-16T11:45:54.725493+0000","last_became_active":"2024-09-16T11:45:39.636105+0000","last_became_peered":"2024-09-16T11:45:39.636105+0000","last_unstale":"2024-09-16T11:45:54.725493+0000","last_undegraded":"2024-09-16T11:45:54.725493+0000","last_fullsized":"2024-09-16T11:45:54.725493+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:18:52.260065+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.014051187999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1102,"num_read_kb":9776,"num_write":200,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"127'334","reported_seq":2413,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703281+0000","last_change":"2024-09-16T11:45:19.592554+0000","last_active":"2024-09-16T11:45:53.703281+0000","last_peered":"2024-09-16T11:45:53.703281+0000","last_clean":"2024-09-16T11:45:53.703281+0000","last_became_active":"2024-09-16T11:43:31.773661+0000","last_became_peered":"2024-09-16T11:43:31.773661+0000","last_unstale":"2024-09-16T11:45:53.703281+0000","last_undegraded":"2024-09-16T11:45:53.703281+0000","last_fullsized":"2024-09-16T11:45:53.703281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:36.062796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052992100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":2191,"num_read_kb":14785,"num_write":375,"num_write_kb":7446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"132'380","reported_seq":2524,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703902+0000","last_change":"2024-09-16T11:45:47.464749+0000","last_active":"2024-09-16T11:45:53.703902+0000","last_peered":"2024-09-16T11:45:53.703902+0000","last_clean":"2024-09-16T11:45:53.703902+0000","last_became_active":"2024-09-16T11:45:47.463209+0000","last_became_peered":"2024-09-16T11:45:47.463209+0000","last_unstale":"2024-09-16T11:45:53.703902+0000","last_undegraded":"2024-09-16T11:45:53.703902+0000","last_fullsized":"2024-09-16T11:45:53.703902+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:29.503448+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0033897049999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3246,"num_read_kb":18406,"num_write":318,"num_write_kb":4718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"132'534","reported_seq":3517,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703670+0000","last_change":"2024-09-16T11:45:47.464540+0000","last_active":"2024-09-16T11:45:53.703670+0000","last_peered":"2024-09-16T11:45:53.703670+0000","last_clean":"2024-09-16T11:45:53.703670+0000","last_became_active":"2024-09-16T11:45:47.463199+0000","last_became_peered":"2024-09-16T11:45:47.463199+0000","last_unstale":"2024-09-16T11:45:53.703670+0000","last_undegraded":"2024-09-16T11:45:53.703670+0000","last_fullsized":"2024-09-16T11:45:53.703670+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:44.772848+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.01209723,"stat_sum":{"num_bytes":19,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":4102,"num_read_kb":12532,"num_write":637,"num_write_kb":9975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"124'658","reported_seq":4607,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725383+0000","last_change":"2024-09-16T11:45:39.678403+0000","last_active":"2024-09-16T11:45:54.725383+0000","last_peered":"2024-09-16T11:45:54.725383+0000","last_clean":"2024-09-16T11:45:54.725383+0000","last_became_active":"2024-09-16T11:45:39.638104+0000","last_became_peered":"2024-09-16T11:45:39.638104+0000","last_unstale":"2024-09-16T11:45:54.725383+0000","last_undegraded":"2024-09-16T11:45:54.725383+0000","last_fullsized":"2024-09-16T11:45:54.725383+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:18.828819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069266390000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5632,"num_read_kb":29340,"num_write":868,"num_write_kb":4351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"127'221","reported_seq":1540,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703616+0000","last_change":"2024-09-16T11:45:19.592170+0000","last_active":"2024-09-16T11:45:53.703616+0000","last_peered":"2024-09-16T11:45:53.703616+0000","last_clean":"2024-09-16T11:45:53.703616+0000","last_became_active":"2024-09-16T11:43:31.776951+0000","last_became_peered":"2024-09-16T11:43:31.776951+0000","last_unstale":"2024-09-16T11:45:53.703616+0000","last_undegraded":"2024-09-16T11:45:53.703616+0000","last_fullsized":"2024-09-16T11:45:53.703616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:28:39.026621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052508100000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1522,"num_read_kb":7050,"num_write":185,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"132'525","reported_seq":2341,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725539+0000","last_change":"2024-09-16T11:45:39.723943+0000","last_active":"2024-09-16T11:45:54.725539+0000","last_peered":"2024-09-16T11:45:54.725539+0000","last_clean":"2024-09-16T11:45:54.725539+0000","last_became_active":"2024-09-16T11:45:39.642890+0000","last_became_peered":"2024-09-16T11:45:39.642890+0000","last_unstale":"2024-09-16T11:45:54.725539+0000","last_undegraded":"2024-09-16T11:45:54.725539+0000","last_fullsized":"2024-09-16T11:45:54.725539+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:26.709287+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062520060000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4283,"num_read_kb":14493,"num_write":541,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"132'743","reported_seq":6417,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566264+0000","last_change":"2024-09-16T11:45:39.656713+0000","last_active":"2024-09-16T11:45:54.566264+0000","last_peered":"2024-09-16T11:45:54.566264+0000","last_clean":"2024-09-16T11:45:54.566264+0000","last_became_active":"2024-09-16T11:45:39.640293+0000","last_became_peered":"2024-09-16T11:45:39.640293+0000","last_unstale":"2024-09-16T11:45:54.566264+0000","last_undegraded":"2024-09-16T11:45:54.566264+0000","last_fullsized":"2024-09-16T11:45:54.566264+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:29:01.940843+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074727580000000004,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8298,"num_read_kb":47234,"num_write":833,"num_write_kb":15319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"127'657","reported_seq":2814,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725859+0000","last_change":"2024-09-16T11:45:19.636993+0000","last_active":"2024-09-16T11:45:54.725859+0000","last_peered":"2024-09-16T11:45:54.725859+0000","last_clean":"2024-09-16T11:45:54.725859+0000","last_became_active":"2024-09-16T11:43:31.889629+0000","last_became_peered":"2024-09-16T11:43:31.889629+0000","last_unstale":"2024-09-16T11:45:54.725859+0000","last_undegraded":"2024-09-16T11:45:54.725859+0000","last_fullsized":"2024-09-16T11:45:54.725859+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:53.124562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030057499999999997,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":3982,"num_read_kb":26490,"num_write":811,"num_write_kb":16305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"132'665","reported_seq":3461,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725390+0000","last_change":"2024-09-16T11:45:39.707640+0000","last_active":"2024-09-16T11:45:54.725390+0000","last_peered":"2024-09-16T11:45:54.725390+0000","last_clean":"2024-09-16T11:45:54.725390+0000","last_became_active":"2024-09-16T11:45:39.641187+0000","last_became_peered":"2024-09-16T11:45:39.641187+0000","last_unstale":"2024-09-16T11:45:54.725390+0000","last_undegraded":"2024-09-16T11:45:54.725390+0000","last_fullsized":"2024-09-16T11:45:54.725390+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:47:39.464422+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074563379999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4363,"num_read_kb":20582,"num_write":914,"num_write_kb":10744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":3197941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"127'546","reported_seq":2511,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703790+0000","last_change":"2024-09-16T11:45:19.592153+0000","last_active":"2024-09-16T11:45:53.703790+0000","last_peered":"2024-09-16T11:45:53.703790+0000","last_clean":"2024-09-16T11:45:53.703790+0000","last_became_active":"2024-09-16T11:43:31.776781+0000","last_became_peered":"2024-09-16T11:43:31.776781+0000","last_unstale":"2024-09-16T11:45:53.703790+0000","last_undegraded":"2024-09-16T11:45:53.703790+0000","last_fullsized":"2024-09-16T11:45:53.703790+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:14.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032933499999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3911,"num_read_kb":15179,"num_write":595,"num_write_kb":10620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"121'294","reported_seq":1908,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703771+0000","last_change":"2024-09-16T11:45:19.642323+0000","last_active":"2024-09-16T11:45:53.703771+0000","last_peered":"2024-09-16T11:45:53.703771+0000","last_clean":"2024-09-16T11:45:53.703771+0000","last_became_active":"2024-09-16T11:43:31.780164+0000","last_became_peered":"2024-09-16T11:43:31.780164+0000","last_unstale":"2024-09-16T11:45:53.703771+0000","last_undegraded":"2024-09-16T11:45:53.703771+0000","last_fullsized":"2024-09-16T11:45:53.703771+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:56.846891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079981,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1687,"num_read_kb":10324,"num_write":299,"num_write_kb":6058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"132'804","reported_seq":2260,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725294+0000","last_change":"2024-09-16T11:45:39.712869+0000","last_active":"2024-09-16T11:45:54.725294+0000","last_peered":"2024-09-16T11:45:54.725294+0000","last_clean":"2024-09-16T11:45:54.725294+0000","last_became_active":"2024-09-16T11:45:39.641126+0000","last_became_peered":"2024-09-16T11:45:39.641126+0000","last_unstale":"2024-09-16T11:45:54.725294+0000","last_undegraded":"2024-09-16T11:45:54.725294+0000","last_fullsized":"2024-09-16T11:45:54.725294+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:23.732386+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074597470000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":4259,"num_read_kb":18254,"num_write":1055,"num_write_kb":13061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"132'401","reported_seq":1908,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725271+0000","last_change":"2024-09-16T11:45:39.666850+0000","last_active":"2024-09-16T11:45:54.725271+0000","last_peered":"2024-09-16T11:45:54.725271+0000","last_clean":"2024-09-16T11:45:54.725271+0000","last_became_active":"2024-09-16T11:45:39.637293+0000","last_became_peered":"2024-09-16T11:45:39.637293+0000","last_unstale":"2024-09-16T11:45:54.725271+0000","last_undegraded":"2024-09-16T11:45:54.725271+0000","last_fullsized":"2024-09-16T11:45:54.725271+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:39.210719+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.016205368000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3249,"num_read_kb":9637,"num_write":346,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725801+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:45:54.725801+0000","last_peered":"2024-09-16T11:45:54.725801+0000","last_clean":"2024-09-16T11:45:54.725801+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:45:54.725801+0000","last_undegraded":"2024-09-16T11:45:54.725801+0000","last_fullsized":"2024-09-16T11:45:54.725801+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"125'347","reported_seq":1761,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703661+0000","last_change":"2024-09-16T11:45:47.465149+0000","last_active":"2024-09-16T11:45:53.703661+0000","last_peered":"2024-09-16T11:45:53.703661+0000","last_clean":"2024-09-16T11:45:53.703661+0000","last_became_active":"2024-09-16T11:45:47.464115+0000","last_became_peered":"2024-09-16T11:45:47.464115+0000","last_unstale":"2024-09-16T11:45:53.703661+0000","last_undegraded":"2024-09-16T11:45:53.703661+0000","last_fullsized":"2024-09-16T11:45:53.703661+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:34.356992+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00038392500000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2491,"num_read_kb":9150,"num_write":283,"num_write_kb":9998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703962+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:45:53.703962+0000","last_peered":"2024-09-16T11:45:53.703962+0000","last_clean":"2024-09-16T11:45:53.703962+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:45:53.703962+0000","last_undegraded":"2024-09-16T11:45:53.703962+0000","last_fullsized":"2024-09-16T11:45:53.703962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"132'477","reported_seq":4581,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725271+0000","last_change":"2024-09-16T11:45:39.723814+0000","last_active":"2024-09-16T11:45:54.725271+0000","last_peered":"2024-09-16T11:45:54.725271+0000","last_clean":"2024-09-16T11:45:54.725271+0000","last_became_active":"2024-09-16T11:45:39.641298+0000","last_became_peered":"2024-09-16T11:45:39.641298+0000","last_unstale":"2024-09-16T11:45:54.725271+0000","last_undegraded":"2024-09-16T11:45:54.725271+0000","last_fullsized":"2024-09-16T11:45:54.725271+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:57.805852+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0094940819999999992,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5544,"num_read_kb":23694,"num_write":515,"num_write_kb":8223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.722985+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:45:53.722985+0000","last_peered":"2024-09-16T11:45:53.722985+0000","last_clean":"2024-09-16T11:45:53.722985+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:45:53.722985+0000","last_undegraded":"2024-09-16T11:45:53.722985+0000","last_fullsized":"2024-09-16T11:45:53.722985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"127'627","reported_seq":6867,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566281+0000","last_change":"2024-09-16T11:45:19.593742+0000","last_active":"2024-09-16T11:45:54.566281+0000","last_peered":"2024-09-16T11:45:54.566281+0000","last_clean":"2024-09-16T11:45:54.566281+0000","last_became_active":"2024-09-16T11:43:31.777896+0000","last_became_peered":"2024-09-16T11:43:31.777896+0000","last_unstale":"2024-09-16T11:45:54.566281+0000","last_undegraded":"2024-09-16T11:45:54.566281+0000","last_fullsized":"2024-09-16T11:45:54.566281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:35.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011770719999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":7410,"num_read_kb":47228,"num_write":617,"num_write_kb":8894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704146+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:45:53.704146+0000","last_peered":"2024-09-16T11:45:53.704146+0000","last_clean":"2024-09-16T11:45:53.704146+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:45:53.704146+0000","last_undegraded":"2024-09-16T11:45:53.704146+0000","last_fullsized":"2024-09-16T11:45:53.704146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"126'382","reported_seq":1083,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703876+0000","last_change":"2024-09-16T11:45:19.642384+0000","last_active":"2024-09-16T11:45:53.703876+0000","last_peered":"2024-09-16T11:45:53.703876+0000","last_clean":"2024-09-16T11:45:53.703876+0000","last_became_active":"2024-09-16T11:43:31.778506+0000","last_became_peered":"2024-09-16T11:43:31.778506+0000","last_unstale":"2024-09-16T11:45:53.703876+0000","last_undegraded":"2024-09-16T11:45:53.703876+0000","last_fullsized":"2024-09-16T11:45:53.703876+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:12.232493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011478079999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1438,"num_read_kb":4478,"num_write":491,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":110,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703900+0000","last_change":"2024-09-16T11:45:47.462953+0000","last_active":"2024-09-16T11:45:53.703900+0000","last_peered":"2024-09-16T11:45:53.703900+0000","last_clean":"2024-09-16T11:45:53.703900+0000","last_became_active":"2024-09-16T11:45:47.462295+0000","last_became_peered":"2024-09-16T11:45:47.462295+0000","last_unstale":"2024-09-16T11:45:53.703900+0000","last_undegraded":"2024-09-16T11:45:53.703900+0000","last_fullsized":"2024-09-16T11:45:53.703900+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:57.786498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":140,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725763+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:45:54.725763+0000","last_peered":"2024-09-16T11:45:54.725763+0000","last_clean":"2024-09-16T11:45:54.725763+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:45:54.725763+0000","last_undegraded":"2024-09-16T11:45:54.725763+0000","last_fullsized":"2024-09-16T11:45:54.725763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"127'734","reported_seq":2603,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704134+0000","last_change":"2024-09-16T11:45:19.641341+0000","last_active":"2024-09-16T11:45:53.704134+0000","last_peered":"2024-09-16T11:45:53.704134+0000","last_clean":"2024-09-16T11:45:53.704134+0000","last_became_active":"2024-09-16T11:43:31.777796+0000","last_became_peered":"2024-09-16T11:43:31.777796+0000","last_unstale":"2024-09-16T11:45:53.704134+0000","last_undegraded":"2024-09-16T11:45:53.704134+0000","last_fullsized":"2024-09-16T11:45:53.704134+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:33.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051098400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4214,"num_read_kb":15199,"num_write":913,"num_write_kb":10139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":132,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704039+0000","last_change":"2024-09-16T11:43:22.440136+0000","last_active":"2024-09-16T11:45:53.704039+0000","last_peered":"2024-09-16T11:45:53.704039+0000","last_clean":"2024-09-16T11:45:53.704039+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:45:53.704039+0000","last_undegraded":"2024-09-16T11:45:53.704039+0000","last_fullsized":"2024-09-16T11:45:53.704039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:18.461948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703187+0000","last_change":"2024-09-16T11:45:39.635580+0000","last_active":"2024-09-16T11:45:53.703187+0000","last_peered":"2024-09-16T11:45:53.703187+0000","last_clean":"2024-09-16T11:45:53.703187+0000","last_became_active":"2024-09-16T11:45:39.635334+0000","last_became_peered":"2024-09-16T11:45:39.635334+0000","last_unstale":"2024-09-16T11:45:53.703187+0000","last_undegraded":"2024-09-16T11:45:53.703187+0000","last_fullsized":"2024-09-16T11:45:53.703187+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:43.033024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"125'546","reported_seq":3390,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704084+0000","last_change":"2024-09-16T11:45:19.641909+0000","last_active":"2024-09-16T11:45:53.704084+0000","last_peered":"2024-09-16T11:45:53.704084+0000","last_clean":"2024-09-16T11:45:53.704084+0000","last_became_active":"2024-09-16T11:43:31.777967+0000","last_became_peered":"2024-09-16T11:43:31.777967+0000","last_unstale":"2024-09-16T11:45:53.704084+0000","last_undegraded":"2024-09-16T11:45:53.704084+0000","last_fullsized":"2024-09-16T11:45:53.704084+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:43.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081483900000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3645,"num_read_kb":23388,"num_write":630,"num_write_kb":9618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"112'2","reported_seq":133,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566166+0000","last_change":"2024-09-16T11:43:22.437965+0000","last_active":"2024-09-16T11:45:54.566166+0000","last_peered":"2024-09-16T11:45:54.566166+0000","last_clean":"2024-09-16T11:45:54.566166+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:45:54.566166+0000","last_undegraded":"2024-09-16T11:45:54.566166+0000","last_fullsized":"2024-09-16T11:45:54.566166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:52.884360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.723066+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:45:53.723066+0000","last_peered":"2024-09-16T11:45:53.723066+0000","last_clean":"2024-09-16T11:45:53.723066+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:45:53.723066+0000","last_undegraded":"2024-09-16T11:45:53.723066+0000","last_fullsized":"2024-09-16T11:45:53.723066+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"127'407","reported_seq":1770,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566334+0000","last_change":"2024-09-16T11:45:19.592986+0000","last_active":"2024-09-16T11:45:54.566334+0000","last_peered":"2024-09-16T11:45:54.566334+0000","last_clean":"2024-09-16T11:45:54.566334+0000","last_became_active":"2024-09-16T11:43:31.776645+0000","last_became_peered":"2024-09-16T11:43:31.776645+0000","last_unstale":"2024-09-16T11:45:54.566334+0000","last_undegraded":"2024-09-16T11:45:54.566334+0000","last_fullsized":"2024-09-16T11:45:54.566334+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:53.198323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066659399999999995,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1650,"num_read_kb":12114,"num_write":331,"num_write_kb":15245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"84'3","reported_seq":97,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725388+0000","last_change":"2024-09-16T11:45:39.638395+0000","last_active":"2024-09-16T11:45:54.725388+0000","last_peered":"2024-09-16T11:45:54.725388+0000","last_clean":"2024-09-16T11:45:54.725388+0000","last_became_active":"2024-09-16T11:45:39.637629+0000","last_became_peered":"2024-09-16T11:45:39.637629+0000","last_unstale":"2024-09-16T11:45:54.725388+0000","last_undegraded":"2024-09-16T11:45:54.725388+0000","last_fullsized":"2024-09-16T11:45:54.725388+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:56.174857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725989+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:45:54.725989+0000","last_peered":"2024-09-16T11:45:54.725989+0000","last_clean":"2024-09-16T11:45:54.725989+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:45:54.725989+0000","last_undegraded":"2024-09-16T11:45:54.725989+0000","last_fullsized":"2024-09-16T11:45:54.725989+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":226,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.723073+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:45:53.723073+0000","last_peered":"2024-09-16T11:45:53.723073+0000","last_clean":"2024-09-16T11:45:53.723073+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:45:53.723073+0000","last_undegraded":"2024-09-16T11:45:53.723073+0000","last_fullsized":"2024-09-16T11:45:53.723073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"125'523","reported_seq":3438,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725834+0000","last_change":"2024-09-16T11:45:19.598172+0000","last_active":"2024-09-16T11:45:54.725834+0000","last_peered":"2024-09-16T11:45:54.725834+0000","last_clean":"2024-09-16T11:45:54.725834+0000","last_became_active":"2024-09-16T11:43:31.782071+0000","last_became_peered":"2024-09-16T11:43:31.782071+0000","last_unstale":"2024-09-16T11:45:54.725834+0000","last_undegraded":"2024-09-16T11:45:54.725834+0000","last_fullsized":"2024-09-16T11:45:54.725834+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:15.516215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000599604,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3801,"num_read_kb":24361,"num_write":540,"num_write_kb":7978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"127'665","reported_seq":9006,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703769+0000","last_change":"2024-09-16T11:45:19.641414+0000","last_active":"2024-09-16T11:45:53.703769+0000","last_peered":"2024-09-16T11:45:53.703769+0000","last_clean":"2024-09-16T11:45:53.703769+0000","last_became_active":"2024-09-16T11:44:10.920131+0000","last_became_peered":"2024-09-16T11:44:10.920131+0000","last_unstale":"2024-09-16T11:45:53.703769+0000","last_undegraded":"2024-09-16T11:45:53.703769+0000","last_fullsized":"2024-09-16T11:45:53.703769+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:55:10.158414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082161099999999998,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":9498,"num_read_kb":69976,"num_write":712,"num_write_kb":19149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":8960357,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"127'878","reported_seq":3861,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703624+0000","last_change":"2024-09-16T11:45:47.465826+0000","last_active":"2024-09-16T11:45:53.703624+0000","last_peered":"2024-09-16T11:45:53.703624+0000","last_clean":"2024-09-16T11:45:53.703624+0000","last_became_active":"2024-09-16T11:45:47.464353+0000","last_became_peered":"2024-09-16T11:45:47.464353+0000","last_unstale":"2024-09-16T11:45:53.703624+0000","last_undegraded":"2024-09-16T11:45:53.703624+0000","last_fullsized":"2024-09-16T11:45:53.703624+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:24.438451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001068741,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":8005,"num_read_kb":12353,"num_write":894,"num_write_kb":6127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"127'623","reported_seq":1817,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.726002+0000","last_change":"2024-09-16T11:45:19.598161+0000","last_active":"2024-09-16T11:45:54.726002+0000","last_peered":"2024-09-16T11:45:54.726002+0000","last_clean":"2024-09-16T11:45:54.726002+0000","last_became_active":"2024-09-16T11:43:31.779384+0000","last_became_peered":"2024-09-16T11:43:31.779384+0000","last_unstale":"2024-09-16T11:45:54.726002+0000","last_undegraded":"2024-09-16T11:45:54.726002+0000","last_fullsized":"2024-09-16T11:45:54.726002+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:34.933146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000647592,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":2523,"num_read_kb":9199,"num_write":666,"num_write_kb":6203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"125'377","reported_seq":3380,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566193+0000","last_change":"2024-09-16T11:45:19.593453+0000","last_active":"2024-09-16T11:45:54.566193+0000","last_peered":"2024-09-16T11:45:54.566193+0000","last_clean":"2024-09-16T11:45:54.566193+0000","last_became_active":"2024-09-16T11:43:31.775335+0000","last_became_peered":"2024-09-16T11:43:31.775335+0000","last_unstale":"2024-09-16T11:45:54.566193+0000","last_undegraded":"2024-09-16T11:45:54.566193+0000","last_fullsized":"2024-09-16T11:45:54.566193+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:07.521713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011352389999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4122,"num_read_kb":15191,"num_write":417,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"132'521","reported_seq":2268,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703740+0000","last_change":"2024-09-16T11:45:47.465501+0000","last_active":"2024-09-16T11:45:53.703740+0000","last_peered":"2024-09-16T11:45:53.703740+0000","last_clean":"2024-09-16T11:45:53.703740+0000","last_became_active":"2024-09-16T11:45:47.463487+0000","last_became_peered":"2024-09-16T11:45:47.463487+0000","last_unstale":"2024-09-16T11:45:53.703740+0000","last_undegraded":"2024-09-16T11:45:53.703740+0000","last_fullsized":"2024-09-16T11:45:53.703740+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:41.597993+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001754511,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4037,"num_read_kb":5871,"num_write":454,"num_write_kb":2926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"125'616","reported_seq":5073,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703823+0000","last_change":"2024-09-16T11:45:47.462731+0000","last_active":"2024-09-16T11:45:53.703823+0000","last_peered":"2024-09-16T11:45:53.703823+0000","last_clean":"2024-09-16T11:45:53.703823+0000","last_became_active":"2024-09-16T11:45:47.461210+0000","last_became_peered":"2024-09-16T11:45:47.461210+0000","last_unstale":"2024-09-16T11:45:53.703823+0000","last_undegraded":"2024-09-16T11:45:53.703823+0000","last_fullsized":"2024-09-16T11:45:53.703823+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:17.838544+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00043040400000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6463,"num_read_kb":36386,"num_write":811,"num_write_kb":14056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"126'605","reported_seq":1931,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703756+0000","last_change":"2024-09-16T11:45:47.465926+0000","last_active":"2024-09-16T11:45:53.703756+0000","last_peered":"2024-09-16T11:45:53.703756+0000","last_clean":"2024-09-16T11:45:53.703756+0000","last_became_active":"2024-09-16T11:45:47.464052+0000","last_became_peered":"2024-09-16T11:45:47.464052+0000","last_unstale":"2024-09-16T11:45:53.703756+0000","last_undegraded":"2024-09-16T11:45:53.703756+0000","last_fullsized":"2024-09-16T11:45:53.703756+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:40.083173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059308799999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2764,"num_read_kb":9887,"num_write":657,"num_write_kb":5333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"127'408","reported_seq":2117,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703980+0000","last_change":"2024-09-16T11:45:39.667068+0000","last_active":"2024-09-16T11:45:53.703980+0000","last_peered":"2024-09-16T11:45:53.703980+0000","last_clean":"2024-09-16T11:45:53.703980+0000","last_became_active":"2024-09-16T11:45:39.643207+0000","last_became_peered":"2024-09-16T11:45:39.643207+0000","last_unstale":"2024-09-16T11:45:53.703980+0000","last_undegraded":"2024-09-16T11:45:53.703980+0000","last_fullsized":"2024-09-16T11:45:53.703980+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:12.700226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070987940000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3609,"num_read_kb":5161,"num_write":362,"num_write_kb":8941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":728,"num_read":135475,"num_read_kb":642896,"num_write":18060,"num_write_kb":290606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},"log_size":16487,"ondisk_log_size":16487,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299128,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072712,"statfs":{"total":96636764160,"available":96330457088,"internally_reserved":0,"allocated":1814528,"data_stored":1481880,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.441,"15min":0.443},"min":{"1min":0.19600000000000001,"5min":0.182,"15min":0.182},"max":{"1min":0.61299999999999999,"5min":0.628,"15min":0.628},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.66000000000000003}]},{"osd":1,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.247,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":4.3559999999999999,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.435,"15min":0.435},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.67900000000000005}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.68400000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.623},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":1.532,"15min":1.532},"last":0.60799999999999998}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.5640000000000001,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.26000000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.67500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45400000000000001,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.437,"15min":0.44},"min":{"1min":0.27600000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65200000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.441},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.318,"5min":0.251,"15min":0.251},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.54700000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.377,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.5},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66200000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52900000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315728,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94056112,"statfs":{"total":96636764160,"available":96313458688,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.438,"15min":0.44400000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52800000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58999999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.503}]},{"osd":1,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.33800000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.53100000000000003,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.439,"15min":0.442},"min":{"1min":0.27400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.61799999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.30399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.35899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.63,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.621}]},{"osd":3,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.55800000000000005}]},{"osd":4,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.64700000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64300000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.48699999999999999,"15min":0.498},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.502,"15min":0.51100000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59899999999999998}]}]},{"osd":3,"up_from":131,"seq":562640715782,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302168,"kb_used_data":1728,"kb_used_omap":43,"kb_used_meta":300372,"kb_avail":94069672,"statfs":{"total":96636764160,"available":96327344128,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44228,"internal_metadata":307581756},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315724,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94056116,"statfs":{"total":96636764160,"available":96313462784,"internally_reserved":0,"allocated":1765376,"data_stored":892439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":1.087,"5min":0.495,"15min":0.39700000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":11.218,"5min":11.218,"15min":11.218},"last":0.31}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32200000000000001},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.372,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.36099999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.38100000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72399999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53500000000000003,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66600000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68600000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53900000000000003}]},{"osd":7,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.871,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311540,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94060300,"statfs":{"total":96636764160,"available":96317747200,"internally_reserved":0,"allocated":1744896,"data_stored":885224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.22700000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.59899999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.379},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.42999999999999999,"15min":0.438},"min":{"1min":0.22900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.61099999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.53700000000000003}]},{"osd":1,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.45300000000000001,"15min":0.45700000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.75800000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47599999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48399999999999999,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":1.026,"15min":1.026},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.47499999999999998,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79600000000000004,"5min":1.01,"15min":1.01},"last":0.56100000000000005}]},{"osd":3,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.622}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39800000000000002,"15min":0.40300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54700000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.502},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.27300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.69899999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.45300000000000001,"15min":0.46500000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.755,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.42099999999999999,"15min":0.433},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52400000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.48199999999999998,"15min":0.495},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67100000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303244,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068596,"statfs":{"total":96636764160,"available":96326242304,"internally_reserved":0,"allocated":1769472,"data_stored":892522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.47599999999999998,"15min":0.441},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":5.8399999999999999,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.44900000000000001,"15min":0.45100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.58699999999999997}]},{"osd":1,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.60699999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.48599999999999999,"15min":0.496},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.621,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.435,"5min":0.49099999999999999,"15min":0.5},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.73499999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.70099999999999996}]},{"osd":5,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.42099999999999999,"15min":0.435},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.44700000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.45500000000000002,"15min":0.46899999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.76400000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.505,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.53500000000000003,"15min":0.55100000000000005},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60299999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.81699999999999995}]},{"osd":7,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56599999999999995,"5min":0.873,"15min":0.873},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58999999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299076,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072764,"statfs":{"total":96636764160,"available":96330510336,"internally_reserved":0,"allocated":1761280,"data_stored":892420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.26200000000000001,"5min":0.29399999999999998,"15min":0.29999999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.46500000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.32800000000000001,"15min":0.32500000000000001},"min":{"1min":0.215,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.255}]},{"osd":1,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32900000000000001,"15min":0.33200000000000002},"min":{"1min":0.23000000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.437,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34799999999999998,"15min":0.34899999999999998},"min":{"1min":0.255,"5min":0.192,"15min":0.192},"max":{"1min":0.53600000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.40400000000000003}]},{"osd":4,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.52700000000000002,"15min":0.54100000000000004},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.35599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.216}]},{"osd":5,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":1.254}]},{"osd":6,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57999999999999996,"15min":0.58899999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":1.288},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.57799999999999996,"15min":0.58799999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.3089999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.266}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299140,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072700,"statfs":{"total":96636764160,"available":96330444800,"internally_reserved":0,"allocated":1822720,"data_stored":1482871,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.39600000000000002,"15min":0.33800000000000002},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":8.5050000000000008,"5min":8.5050000000000008,"15min":8.5050000000000008},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32100000000000001},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.436,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.26200000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.39600000000000002,"15min":0.33600000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":8.4689999999999994,"5min":8.4689999999999994,"15min":8.4689999999999994},"last":0.315},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.23699999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47699999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.30399999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.27700000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.57399999999999995,"15min":0.51400000000000001},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":8.5299999999999994,"5min":8.5299999999999994,"15min":8.5299999999999994},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.503,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71199999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.496}]},{"osd":5,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.65100000000000002,"15min":0.60299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":8.6039999999999992,"5min":8.6039999999999992,"15min":8.6039999999999992},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.55300000000000005,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69099999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.39500000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.64200000000000002,"5min":1.075,"15min":1.075},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.67500000000000004,"15min":0.628},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":8.5809999999999995,"5min":8.5809999999999995,"15min":8.5809999999999995},"last":0.46800000000000003}]},{"osd":7,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64800000000000002,"5min":0.747,"15min":0.747},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.61199999999999999,"15min":0.55200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":8.5570000000000004,"5min":8.5570000000000004,"15min":8.5570000000000004},"last":0.42399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:45:59.477 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:45:59.736 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:45:59.737 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:45:59.747 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-16T11:45:59.402327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590574,"num_objects":741,"num_object_clones":0,"num_object_copies":1482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":731,"num_read":135541,"num_read_kb":642951,"num_write":18165,"num_write_kb":292238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16567,"ondisk_log_size":16567,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445748,"kb_used_data":13884,"kb_used_omap":54,"kb_used_meta":2431689,"kb_avail":752528972,"statfs":{"total":773094113280,"available":770589667328,"internally_reserved":0,"allocated":14217216,"data_stored":8312404,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56156,"internal_metadata":2490049700},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.705432"},"pg_stats":[{"pgid":"4.1f","version":"127'327","reported_seq":6237,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725885+0000","last_change":"2024-09-16T11:45:19.597844+0000","last_active":"2024-09-16T11:45:54.725885+0000","last_peered":"2024-09-16T11:45:54.725885+0000","last_clean":"2024-09-16T11:45:54.725885+0000","last_became_active":"2024-09-16T11:43:31.779428+0000","last_became_peered":"2024-09-16T11:43:31.779428+0000","last_unstale":"2024-09-16T11:45:54.725885+0000","last_undegraded":"2024-09-16T11:45:54.725885+0000","last_fullsized":"2024-09-16T11:45:54.725885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:30.437698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034041099999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6762,"num_read_kb":29123,"num_write":304,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"125'449","reported_seq":5346,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566266+0000","last_change":"2024-09-16T11:45:19.592718+0000","last_active":"2024-09-16T11:45:54.566266+0000","last_peered":"2024-09-16T11:45:54.566266+0000","last_clean":"2024-09-16T11:45:54.566266+0000","last_became_active":"2024-09-16T11:43:31.889706+0000","last_became_peered":"2024-09-16T11:43:31.889706+0000","last_unstale":"2024-09-16T11:45:54.566266+0000","last_undegraded":"2024-09-16T11:45:54.566266+0000","last_fullsized":"2024-09-16T11:45:54.566266+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:25.242116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000654113,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5672,"num_read_kb":36055,"num_write":486,"num_write_kb":9211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"132'213","reported_seq":1539,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725493+0000","last_change":"2024-09-16T11:45:39.661471+0000","last_active":"2024-09-16T11:45:54.725493+0000","last_peered":"2024-09-16T11:45:54.725493+0000","last_clean":"2024-09-16T11:45:54.725493+0000","last_became_active":"2024-09-16T11:45:39.636105+0000","last_became_peered":"2024-09-16T11:45:39.636105+0000","last_unstale":"2024-09-16T11:45:54.725493+0000","last_undegraded":"2024-09-16T11:45:54.725493+0000","last_fullsized":"2024-09-16T11:45:54.725493+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:18:52.260065+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.014051187999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1102,"num_read_kb":9776,"num_write":200,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"127'334","reported_seq":2413,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703281+0000","last_change":"2024-09-16T11:45:19.592554+0000","last_active":"2024-09-16T11:45:53.703281+0000","last_peered":"2024-09-16T11:45:53.703281+0000","last_clean":"2024-09-16T11:45:53.703281+0000","last_became_active":"2024-09-16T11:43:31.773661+0000","last_became_peered":"2024-09-16T11:43:31.773661+0000","last_unstale":"2024-09-16T11:45:53.703281+0000","last_undegraded":"2024-09-16T11:45:53.703281+0000","last_fullsized":"2024-09-16T11:45:53.703281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:36.062796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052992100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":2191,"num_read_kb":14785,"num_write":375,"num_write_kb":7446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"132'380","reported_seq":2524,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703902+0000","last_change":"2024-09-16T11:45:47.464749+0000","last_active":"2024-09-16T11:45:53.703902+0000","last_peered":"2024-09-16T11:45:53.703902+0000","last_clean":"2024-09-16T11:45:53.703902+0000","last_became_active":"2024-09-16T11:45:47.463209+0000","last_became_peered":"2024-09-16T11:45:47.463209+0000","last_unstale":"2024-09-16T11:45:53.703902+0000","last_undegraded":"2024-09-16T11:45:53.703902+0000","last_fullsized":"2024-09-16T11:45:53.703902+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:29.503448+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0033897049999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3246,"num_read_kb":18406,"num_write":318,"num_write_kb":4718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"132'534","reported_seq":3517,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703670+0000","last_change":"2024-09-16T11:45:47.464540+0000","last_active":"2024-09-16T11:45:53.703670+0000","last_peered":"2024-09-16T11:45:53.703670+0000","last_clean":"2024-09-16T11:45:53.703670+0000","last_became_active":"2024-09-16T11:45:47.463199+0000","last_became_peered":"2024-09-16T11:45:47.463199+0000","last_unstale":"2024-09-16T11:45:53.703670+0000","last_undegraded":"2024-09-16T11:45:53.703670+0000","last_fullsized":"2024-09-16T11:45:53.703670+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:44.772848+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.01209723,"stat_sum":{"num_bytes":19,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":4102,"num_read_kb":12532,"num_write":637,"num_write_kb":9975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"124'658","reported_seq":4607,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725383+0000","last_change":"2024-09-16T11:45:39.678403+0000","last_active":"2024-09-16T11:45:54.725383+0000","last_peered":"2024-09-16T11:45:54.725383+0000","last_clean":"2024-09-16T11:45:54.725383+0000","last_became_active":"2024-09-16T11:45:39.638104+0000","last_became_peered":"2024-09-16T11:45:39.638104+0000","last_unstale":"2024-09-16T11:45:54.725383+0000","last_undegraded":"2024-09-16T11:45:54.725383+0000","last_fullsized":"2024-09-16T11:45:54.725383+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:18.828819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069266390000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5632,"num_read_kb":29340,"num_write":868,"num_write_kb":4351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"127'221","reported_seq":1540,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703616+0000","last_change":"2024-09-16T11:45:19.592170+0000","last_active":"2024-09-16T11:45:53.703616+0000","last_peered":"2024-09-16T11:45:53.703616+0000","last_clean":"2024-09-16T11:45:53.703616+0000","last_became_active":"2024-09-16T11:43:31.776951+0000","last_became_peered":"2024-09-16T11:43:31.776951+0000","last_unstale":"2024-09-16T11:45:53.703616+0000","last_undegraded":"2024-09-16T11:45:53.703616+0000","last_fullsized":"2024-09-16T11:45:53.703616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:28:39.026621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052508100000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1522,"num_read_kb":7050,"num_write":185,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"132'525","reported_seq":2341,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725539+0000","last_change":"2024-09-16T11:45:39.723943+0000","last_active":"2024-09-16T11:45:54.725539+0000","last_peered":"2024-09-16T11:45:54.725539+0000","last_clean":"2024-09-16T11:45:54.725539+0000","last_became_active":"2024-09-16T11:45:39.642890+0000","last_became_peered":"2024-09-16T11:45:39.642890+0000","last_unstale":"2024-09-16T11:45:54.725539+0000","last_undegraded":"2024-09-16T11:45:54.725539+0000","last_fullsized":"2024-09-16T11:45:54.725539+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:26.709287+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062520060000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4283,"num_read_kb":14493,"num_write":541,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"132'743","reported_seq":6417,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566264+0000","last_change":"2024-09-16T11:45:39.656713+0000","last_active":"2024-09-16T11:45:54.566264+0000","last_peered":"2024-09-16T11:45:54.566264+0000","last_clean":"2024-09-16T11:45:54.566264+0000","last_became_active":"2024-09-16T11:45:39.640293+0000","last_became_peered":"2024-09-16T11:45:39.640293+0000","last_unstale":"2024-09-16T11:45:54.566264+0000","last_undegraded":"2024-09-16T11:45:54.566264+0000","last_fullsized":"2024-09-16T11:45:54.566264+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:29:01.940843+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074727580000000004,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8298,"num_read_kb":47234,"num_write":833,"num_write_kb":15319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"127'657","reported_seq":2814,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725859+0000","last_change":"2024-09-16T11:45:19.636993+0000","last_active":"2024-09-16T11:45:54.725859+0000","last_peered":"2024-09-16T11:45:54.725859+0000","last_clean":"2024-09-16T11:45:54.725859+0000","last_became_active":"2024-09-16T11:43:31.889629+0000","last_became_peered":"2024-09-16T11:43:31.889629+0000","last_unstale":"2024-09-16T11:45:54.725859+0000","last_undegraded":"2024-09-16T11:45:54.725859+0000","last_fullsized":"2024-09-16T11:45:54.725859+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:53.124562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030057499999999997,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":3982,"num_read_kb":26490,"num_write":811,"num_write_kb":16305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"132'665","reported_seq":3461,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725390+0000","last_change":"2024-09-16T11:45:39.707640+0000","last_active":"2024-09-16T11:45:54.725390+0000","last_peered":"2024-09-16T11:45:54.725390+0000","last_clean":"2024-09-16T11:45:54.725390+0000","last_became_active":"2024-09-16T11:45:39.641187+0000","last_became_peered":"2024-09-16T11:45:39.641187+0000","last_unstale":"2024-09-16T11:45:54.725390+0000","last_undegraded":"2024-09-16T11:45:54.725390+0000","last_fullsized":"2024-09-16T11:45:54.725390+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:47:39.464422+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074563379999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4363,"num_read_kb":20582,"num_write":914,"num_write_kb":10744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":3197941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"127'546","reported_seq":2511,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703790+0000","last_change":"2024-09-16T11:45:19.592153+0000","last_active":"2024-09-16T11:45:53.703790+0000","last_peered":"2024-09-16T11:45:53.703790+0000","last_clean":"2024-09-16T11:45:53.703790+0000","last_became_active":"2024-09-16T11:43:31.776781+0000","last_became_peered":"2024-09-16T11:43:31.776781+0000","last_unstale":"2024-09-16T11:45:53.703790+0000","last_undegraded":"2024-09-16T11:45:53.703790+0000","last_fullsized":"2024-09-16T11:45:53.703790+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:14.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032933499999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3911,"num_read_kb":15179,"num_write":595,"num_write_kb":10620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"121'294","reported_seq":1908,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703771+0000","last_change":"2024-09-16T11:45:19.642323+0000","last_active":"2024-09-16T11:45:53.703771+0000","last_peered":"2024-09-16T11:45:53.703771+0000","last_clean":"2024-09-16T11:45:53.703771+0000","last_became_active":"2024-09-16T11:43:31.780164+0000","last_became_peered":"2024-09-16T11:43:31.780164+0000","last_unstale":"2024-09-16T11:45:53.703771+0000","last_undegraded":"2024-09-16T11:45:53.703771+0000","last_fullsized":"2024-09-16T11:45:53.703771+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:56.846891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079981,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1687,"num_read_kb":10324,"num_write":299,"num_write_kb":6058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"132'804","reported_seq":2260,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725294+0000","last_change":"2024-09-16T11:45:39.712869+0000","last_active":"2024-09-16T11:45:54.725294+0000","last_peered":"2024-09-16T11:45:54.725294+0000","last_clean":"2024-09-16T11:45:54.725294+0000","last_became_active":"2024-09-16T11:45:39.641126+0000","last_became_peered":"2024-09-16T11:45:39.641126+0000","last_unstale":"2024-09-16T11:45:54.725294+0000","last_undegraded":"2024-09-16T11:45:54.725294+0000","last_fullsized":"2024-09-16T11:45:54.725294+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:23.732386+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074597470000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":4259,"num_read_kb":18254,"num_write":1055,"num_write_kb":13061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"132'401","reported_seq":1908,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725271+0000","last_change":"2024-09-16T11:45:39.666850+0000","last_active":"2024-09-16T11:45:54.725271+0000","last_peered":"2024-09-16T11:45:54.725271+0000","last_clean":"2024-09-16T11:45:54.725271+0000","last_became_active":"2024-09-16T11:45:39.637293+0000","last_became_peered":"2024-09-16T11:45:39.637293+0000","last_unstale":"2024-09-16T11:45:54.725271+0000","last_undegraded":"2024-09-16T11:45:54.725271+0000","last_fullsized":"2024-09-16T11:45:54.725271+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:39.210719+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.016205368000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3249,"num_read_kb":9637,"num_write":346,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725801+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:45:54.725801+0000","last_peered":"2024-09-16T11:45:54.725801+0000","last_clean":"2024-09-16T11:45:54.725801+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:45:54.725801+0000","last_undegraded":"2024-09-16T11:45:54.725801+0000","last_fullsized":"2024-09-16T11:45:54.725801+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"125'347","reported_seq":1761,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703661+0000","last_change":"2024-09-16T11:45:47.465149+0000","last_active":"2024-09-16T11:45:53.703661+0000","last_peered":"2024-09-16T11:45:53.703661+0000","last_clean":"2024-09-16T11:45:53.703661+0000","last_became_active":"2024-09-16T11:45:47.464115+0000","last_became_peered":"2024-09-16T11:45:47.464115+0000","last_unstale":"2024-09-16T11:45:53.703661+0000","last_undegraded":"2024-09-16T11:45:53.703661+0000","last_fullsized":"2024-09-16T11:45:53.703661+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:34.356992+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00038392500000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2491,"num_read_kb":9150,"num_write":283,"num_write_kb":9998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703962+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:45:53.703962+0000","last_peered":"2024-09-16T11:45:53.703962+0000","last_clean":"2024-09-16T11:45:53.703962+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:45:53.703962+0000","last_undegraded":"2024-09-16T11:45:53.703962+0000","last_fullsized":"2024-09-16T11:45:53.703962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"132'477","reported_seq":4581,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725271+0000","last_change":"2024-09-16T11:45:39.723814+0000","last_active":"2024-09-16T11:45:54.725271+0000","last_peered":"2024-09-16T11:45:54.725271+0000","last_clean":"2024-09-16T11:45:54.725271+0000","last_became_active":"2024-09-16T11:45:39.641298+0000","last_became_peered":"2024-09-16T11:45:39.641298+0000","last_unstale":"2024-09-16T11:45:54.725271+0000","last_undegraded":"2024-09-16T11:45:54.725271+0000","last_fullsized":"2024-09-16T11:45:54.725271+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:57.805852+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0094940819999999992,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5544,"num_read_kb":23694,"num_write":515,"num_write_kb":8223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.722985+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:45:53.722985+0000","last_peered":"2024-09-16T11:45:53.722985+0000","last_clean":"2024-09-16T11:45:53.722985+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:45:53.722985+0000","last_undegraded":"2024-09-16T11:45:53.722985+0000","last_fullsized":"2024-09-16T11:45:53.722985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"127'627","reported_seq":6867,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566281+0000","last_change":"2024-09-16T11:45:19.593742+0000","last_active":"2024-09-16T11:45:54.566281+0000","last_peered":"2024-09-16T11:45:54.566281+0000","last_clean":"2024-09-16T11:45:54.566281+0000","last_became_active":"2024-09-16T11:43:31.777896+0000","last_became_peered":"2024-09-16T11:43:31.777896+0000","last_unstale":"2024-09-16T11:45:54.566281+0000","last_undegraded":"2024-09-16T11:45:54.566281+0000","last_fullsized":"2024-09-16T11:45:54.566281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:35.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011770719999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":7410,"num_read_kb":47228,"num_write":617,"num_write_kb":8894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704146+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:45:53.704146+0000","last_peered":"2024-09-16T11:45:53.704146+0000","last_clean":"2024-09-16T11:45:53.704146+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:45:53.704146+0000","last_undegraded":"2024-09-16T11:45:53.704146+0000","last_fullsized":"2024-09-16T11:45:53.704146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"126'382","reported_seq":1083,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703876+0000","last_change":"2024-09-16T11:45:19.642384+0000","last_active":"2024-09-16T11:45:53.703876+0000","last_peered":"2024-09-16T11:45:53.703876+0000","last_clean":"2024-09-16T11:45:53.703876+0000","last_became_active":"2024-09-16T11:43:31.778506+0000","last_became_peered":"2024-09-16T11:43:31.778506+0000","last_unstale":"2024-09-16T11:45:53.703876+0000","last_undegraded":"2024-09-16T11:45:53.703876+0000","last_fullsized":"2024-09-16T11:45:53.703876+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:12.232493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011478079999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1438,"num_read_kb":4478,"num_write":491,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":110,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703900+0000","last_change":"2024-09-16T11:45:47.462953+0000","last_active":"2024-09-16T11:45:53.703900+0000","last_peered":"2024-09-16T11:45:53.703900+0000","last_clean":"2024-09-16T11:45:53.703900+0000","last_became_active":"2024-09-16T11:45:47.462295+0000","last_became_peered":"2024-09-16T11:45:47.462295+0000","last_unstale":"2024-09-16T11:45:53.703900+0000","last_undegraded":"2024-09-16T11:45:53.703900+0000","last_fullsized":"2024-09-16T11:45:53.703900+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:57.786498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":140,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725763+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:45:54.725763+0000","last_peered":"2024-09-16T11:45:54.725763+0000","last_clean":"2024-09-16T11:45:54.725763+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:45:54.725763+0000","last_undegraded":"2024-09-16T11:45:54.725763+0000","last_fullsized":"2024-09-16T11:45:54.725763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"127'734","reported_seq":2603,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704134+0000","last_change":"2024-09-16T11:45:19.641341+0000","last_active":"2024-09-16T11:45:53.704134+0000","last_peered":"2024-09-16T11:45:53.704134+0000","last_clean":"2024-09-16T11:45:53.704134+0000","last_became_active":"2024-09-16T11:43:31.777796+0000","last_became_peered":"2024-09-16T11:43:31.777796+0000","last_unstale":"2024-09-16T11:45:53.704134+0000","last_undegraded":"2024-09-16T11:45:53.704134+0000","last_fullsized":"2024-09-16T11:45:53.704134+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:33.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051098400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4214,"num_read_kb":15199,"num_write":913,"num_write_kb":10139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":132,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704039+0000","last_change":"2024-09-16T11:43:22.440136+0000","last_active":"2024-09-16T11:45:53.704039+0000","last_peered":"2024-09-16T11:45:53.704039+0000","last_clean":"2024-09-16T11:45:53.704039+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:45:53.704039+0000","last_undegraded":"2024-09-16T11:45:53.704039+0000","last_fullsized":"2024-09-16T11:45:53.704039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:18.461948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703187+0000","last_change":"2024-09-16T11:45:39.635580+0000","last_active":"2024-09-16T11:45:53.703187+0000","last_peered":"2024-09-16T11:45:53.703187+0000","last_clean":"2024-09-16T11:45:53.703187+0000","last_became_active":"2024-09-16T11:45:39.635334+0000","last_became_peered":"2024-09-16T11:45:39.635334+0000","last_unstale":"2024-09-16T11:45:53.703187+0000","last_undegraded":"2024-09-16T11:45:53.703187+0000","last_fullsized":"2024-09-16T11:45:53.703187+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:43.033024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"125'546","reported_seq":3390,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704084+0000","last_change":"2024-09-16T11:45:19.641909+0000","last_active":"2024-09-16T11:45:53.704084+0000","last_peered":"2024-09-16T11:45:53.704084+0000","last_clean":"2024-09-16T11:45:53.704084+0000","last_became_active":"2024-09-16T11:43:31.777967+0000","last_became_peered":"2024-09-16T11:43:31.777967+0000","last_unstale":"2024-09-16T11:45:53.704084+0000","last_undegraded":"2024-09-16T11:45:53.704084+0000","last_fullsized":"2024-09-16T11:45:53.704084+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:43.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081483900000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3645,"num_read_kb":23388,"num_write":630,"num_write_kb":9618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"112'2","reported_seq":133,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566166+0000","last_change":"2024-09-16T11:43:22.437965+0000","last_active":"2024-09-16T11:45:54.566166+0000","last_peered":"2024-09-16T11:45:54.566166+0000","last_clean":"2024-09-16T11:45:54.566166+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:45:54.566166+0000","last_undegraded":"2024-09-16T11:45:54.566166+0000","last_fullsized":"2024-09-16T11:45:54.566166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:52.884360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.723066+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:45:53.723066+0000","last_peered":"2024-09-16T11:45:53.723066+0000","last_clean":"2024-09-16T11:45:53.723066+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:45:53.723066+0000","last_undegraded":"2024-09-16T11:45:53.723066+0000","last_fullsized":"2024-09-16T11:45:53.723066+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"127'407","reported_seq":1770,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566334+0000","last_change":"2024-09-16T11:45:19.592986+0000","last_active":"2024-09-16T11:45:54.566334+0000","last_peered":"2024-09-16T11:45:54.566334+0000","last_clean":"2024-09-16T11:45:54.566334+0000","last_became_active":"2024-09-16T11:43:31.776645+0000","last_became_peered":"2024-09-16T11:43:31.776645+0000","last_unstale":"2024-09-16T11:45:54.566334+0000","last_undegraded":"2024-09-16T11:45:54.566334+0000","last_fullsized":"2024-09-16T11:45:54.566334+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:53.198323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066659399999999995,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1650,"num_read_kb":12114,"num_write":331,"num_write_kb":15245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"84'3","reported_seq":97,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725388+0000","last_change":"2024-09-16T11:45:39.638395+0000","last_active":"2024-09-16T11:45:54.725388+0000","last_peered":"2024-09-16T11:45:54.725388+0000","last_clean":"2024-09-16T11:45:54.725388+0000","last_became_active":"2024-09-16T11:45:39.637629+0000","last_became_peered":"2024-09-16T11:45:39.637629+0000","last_unstale":"2024-09-16T11:45:54.725388+0000","last_undegraded":"2024-09-16T11:45:54.725388+0000","last_fullsized":"2024-09-16T11:45:54.725388+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:56.174857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725989+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:45:54.725989+0000","last_peered":"2024-09-16T11:45:54.725989+0000","last_clean":"2024-09-16T11:45:54.725989+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:45:54.725989+0000","last_undegraded":"2024-09-16T11:45:54.725989+0000","last_fullsized":"2024-09-16T11:45:54.725989+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":226,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.723073+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:45:53.723073+0000","last_peered":"2024-09-16T11:45:53.723073+0000","last_clean":"2024-09-16T11:45:53.723073+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:45:53.723073+0000","last_undegraded":"2024-09-16T11:45:53.723073+0000","last_fullsized":"2024-09-16T11:45:53.723073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"125'523","reported_seq":3438,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725834+0000","last_change":"2024-09-16T11:45:19.598172+0000","last_active":"2024-09-16T11:45:54.725834+0000","last_peered":"2024-09-16T11:45:54.725834+0000","last_clean":"2024-09-16T11:45:54.725834+0000","last_became_active":"2024-09-16T11:43:31.782071+0000","last_became_peered":"2024-09-16T11:43:31.782071+0000","last_unstale":"2024-09-16T11:45:54.725834+0000","last_undegraded":"2024-09-16T11:45:54.725834+0000","last_fullsized":"2024-09-16T11:45:54.725834+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:15.516215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000599604,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3801,"num_read_kb":24361,"num_write":540,"num_write_kb":7978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"127'665","reported_seq":9006,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703769+0000","last_change":"2024-09-16T11:45:19.641414+0000","last_active":"2024-09-16T11:45:53.703769+0000","last_peered":"2024-09-16T11:45:53.703769+0000","last_clean":"2024-09-16T11:45:53.703769+0000","last_became_active":"2024-09-16T11:44:10.920131+0000","last_became_peered":"2024-09-16T11:44:10.920131+0000","last_unstale":"2024-09-16T11:45:53.703769+0000","last_undegraded":"2024-09-16T11:45:53.703769+0000","last_fullsized":"2024-09-16T11:45:53.703769+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:55:10.158414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082161099999999998,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":9498,"num_read_kb":69976,"num_write":712,"num_write_kb":19149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":8960357,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"127'878","reported_seq":3861,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703624+0000","last_change":"2024-09-16T11:45:47.465826+0000","last_active":"2024-09-16T11:45:53.703624+0000","last_peered":"2024-09-16T11:45:53.703624+0000","last_clean":"2024-09-16T11:45:53.703624+0000","last_became_active":"2024-09-16T11:45:47.464353+0000","last_became_peered":"2024-09-16T11:45:47.464353+0000","last_unstale":"2024-09-16T11:45:53.703624+0000","last_undegraded":"2024-09-16T11:45:53.703624+0000","last_fullsized":"2024-09-16T11:45:53.703624+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:24.438451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001068741,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":8005,"num_read_kb":12353,"num_write":894,"num_write_kb":6127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"127'623","reported_seq":1817,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.726002+0000","last_change":"2024-09-16T11:45:19.598161+0000","last_active":"2024-09-16T11:45:54.726002+0000","last_peered":"2024-09-16T11:45:54.726002+0000","last_clean":"2024-09-16T11:45:54.726002+0000","last_became_active":"2024-09-16T11:43:31.779384+0000","last_became_peered":"2024-09-16T11:43:31.779384+0000","last_unstale":"2024-09-16T11:45:54.726002+0000","last_undegraded":"2024-09-16T11:45:54.726002+0000","last_fullsized":"2024-09-16T11:45:54.726002+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:34.933146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000647592,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":2523,"num_read_kb":9199,"num_write":666,"num_write_kb":6203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"125'377","reported_seq":3380,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566193+0000","last_change":"2024-09-16T11:45:19.593453+0000","last_active":"2024-09-16T11:45:54.566193+0000","last_peered":"2024-09-16T11:45:54.566193+0000","last_clean":"2024-09-16T11:45:54.566193+0000","last_became_active":"2024-09-16T11:43:31.775335+0000","last_became_peered":"2024-09-16T11:43:31.775335+0000","last_unstale":"2024-09-16T11:45:54.566193+0000","last_undegraded":"2024-09-16T11:45:54.566193+0000","last_fullsized":"2024-09-16T11:45:54.566193+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:07.521713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011352389999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4122,"num_read_kb":15191,"num_write":417,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"132'521","reported_seq":2268,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703740+0000","last_change":"2024-09-16T11:45:47.465501+0000","last_active":"2024-09-16T11:45:53.703740+0000","last_peered":"2024-09-16T11:45:53.703740+0000","last_clean":"2024-09-16T11:45:53.703740+0000","last_became_active":"2024-09-16T11:45:47.463487+0000","last_became_peered":"2024-09-16T11:45:47.463487+0000","last_unstale":"2024-09-16T11:45:53.703740+0000","last_undegraded":"2024-09-16T11:45:53.703740+0000","last_fullsized":"2024-09-16T11:45:53.703740+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:41.597993+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001754511,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4037,"num_read_kb":5871,"num_write":454,"num_write_kb":2926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"125'616","reported_seq":5073,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703823+0000","last_change":"2024-09-16T11:45:47.462731+0000","last_active":"2024-09-16T11:45:53.703823+0000","last_peered":"2024-09-16T11:45:53.703823+0000","last_clean":"2024-09-16T11:45:53.703823+0000","last_became_active":"2024-09-16T11:45:47.461210+0000","last_became_peered":"2024-09-16T11:45:47.461210+0000","last_unstale":"2024-09-16T11:45:53.703823+0000","last_undegraded":"2024-09-16T11:45:53.703823+0000","last_fullsized":"2024-09-16T11:45:53.703823+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:17.838544+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00043040400000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6463,"num_read_kb":36386,"num_write":811,"num_write_kb":14056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"126'605","reported_seq":1931,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703756+0000","last_change":"2024-09-16T11:45:47.465926+0000","last_active":"2024-09-16T11:45:53.703756+0000","last_peered":"2024-09-16T11:45:53.703756+0000","last_clean":"2024-09-16T11:45:53.703756+0000","last_became_active":"2024-09-16T11:45:47.464052+0000","last_became_peered":"2024-09-16T11:45:47.464052+0000","last_unstale":"2024-09-16T11:45:53.703756+0000","last_undegraded":"2024-09-16T11:45:53.703756+0000","last_fullsized":"2024-09-16T11:45:53.703756+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:40.083173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059308799999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2764,"num_read_kb":9887,"num_write":657,"num_write_kb":5333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"127'408","reported_seq":2117,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703980+0000","last_change":"2024-09-16T11:45:39.667068+0000","last_active":"2024-09-16T11:45:53.703980+0000","last_peered":"2024-09-16T11:45:53.703980+0000","last_clean":"2024-09-16T11:45:53.703980+0000","last_became_active":"2024-09-16T11:45:39.643207+0000","last_became_peered":"2024-09-16T11:45:39.643207+0000","last_unstale":"2024-09-16T11:45:53.703980+0000","last_undegraded":"2024-09-16T11:45:53.703980+0000","last_fullsized":"2024-09-16T11:45:53.703980+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:12.700226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070987940000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3609,"num_read_kb":5161,"num_write":362,"num_write_kb":8941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":728,"num_read":135475,"num_read_kb":642896,"num_write":18060,"num_write_kb":290606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},"log_size":16487,"ondisk_log_size":16487,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299128,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072712,"statfs":{"total":96636764160,"available":96330457088,"internally_reserved":0,"allocated":1814528,"data_stored":1481880,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.441,"15min":0.443},"min":{"1min":0.19600000000000001,"5min":0.182,"15min":0.182},"max":{"1min":0.61299999999999999,"5min":0.628,"15min":0.628},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.66000000000000003}]},{"osd":1,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.247,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":4.3559999999999999,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.435,"15min":0.435},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.67900000000000005}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.68400000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.623},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":1.532,"15min":1.532},"last":0.60799999999999998}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.5640000000000001,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.26000000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.67500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45400000000000001,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.437,"15min":0.44},"min":{"1min":0.27600000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65200000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.441},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.318,"5min":0.251,"15min":0.251},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.54700000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.377,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.5},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66200000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52900000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315728,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94056112,"statfs":{"total":96636764160,"available":96313458688,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.438,"15min":0.44400000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52800000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58999999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.503}]},{"osd":1,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.33800000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.53100000000000003,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.439,"15min":0.442},"min":{"1min":0.27400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.61799999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.30399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.35899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.63,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.621}]},{"osd":3,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.55800000000000005}]},{"osd":4,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.64700000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64300000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.48699999999999999,"15min":0.498},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.502,"15min":0.51100000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59899999999999998}]}]},{"osd":3,"up_from":131,"seq":562640715782,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302168,"kb_used_data":1728,"kb_used_omap":43,"kb_used_meta":300372,"kb_avail":94069672,"statfs":{"total":96636764160,"available":96327344128,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44228,"internal_metadata":307581756},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315724,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94056116,"statfs":{"total":96636764160,"available":96313462784,"internally_reserved":0,"allocated":1765376,"data_stored":892439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":1.087,"5min":0.495,"15min":0.39700000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":11.218,"5min":11.218,"15min":11.218},"last":0.31}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32200000000000001},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.372,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.36099999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.38100000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72399999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53500000000000003,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66600000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68600000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53900000000000003}]},{"osd":7,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.871,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311540,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94060300,"statfs":{"total":96636764160,"available":96317747200,"internally_reserved":0,"allocated":1744896,"data_stored":885224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.22700000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.59899999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.379},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.42999999999999999,"15min":0.438},"min":{"1min":0.22900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.61099999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.53700000000000003}]},{"osd":1,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.45300000000000001,"15min":0.45700000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.75800000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47599999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48399999999999999,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":1.026,"15min":1.026},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.47499999999999998,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79600000000000004,"5min":1.01,"15min":1.01},"last":0.56100000000000005}]},{"osd":3,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.622}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39800000000000002,"15min":0.40300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54700000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.502},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.27300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.69899999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.45300000000000001,"15min":0.46500000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.755,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.42099999999999999,"15min":0.433},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52400000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.48199999999999998,"15min":0.495},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67100000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303244,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068596,"statfs":{"total":96636764160,"available":96326242304,"internally_reserved":0,"allocated":1769472,"data_stored":892522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.47599999999999998,"15min":0.441},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":5.8399999999999999,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.44900000000000001,"15min":0.45100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.58699999999999997}]},{"osd":1,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.60699999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.48599999999999999,"15min":0.496},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.621,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.435,"5min":0.49099999999999999,"15min":0.5},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.73499999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.70099999999999996}]},{"osd":5,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.42099999999999999,"15min":0.435},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.44700000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.45500000000000002,"15min":0.46899999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.76400000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.505,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.53500000000000003,"15min":0.55100000000000005},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60299999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.81699999999999995}]},{"osd":7,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56599999999999995,"5min":0.873,"15min":0.873},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58999999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299076,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072764,"statfs":{"total":96636764160,"available":96330510336,"internally_reserved":0,"allocated":1761280,"data_stored":892420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.26200000000000001,"5min":0.29399999999999998,"15min":0.29999999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.46500000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.32800000000000001,"15min":0.32500000000000001},"min":{"1min":0.215,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.255}]},{"osd":1,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32900000000000001,"15min":0.33200000000000002},"min":{"1min":0.23000000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.437,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34799999999999998,"15min":0.34899999999999998},"min":{"1min":0.255,"5min":0.192,"15min":0.192},"max":{"1min":0.53600000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.40400000000000003}]},{"osd":4,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.52700000000000002,"15min":0.54100000000000004},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.35599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.216}]},{"osd":5,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":1.254}]},{"osd":6,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57999999999999996,"15min":0.58899999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":1.288},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.57799999999999996,"15min":0.58799999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.3089999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.266}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299140,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072700,"statfs":{"total":96636764160,"available":96330444800,"internally_reserved":0,"allocated":1822720,"data_stored":1482871,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.39600000000000002,"15min":0.33800000000000002},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":8.5050000000000008,"5min":8.5050000000000008,"15min":8.5050000000000008},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32100000000000001},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.436,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.26200000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.39600000000000002,"15min":0.33600000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":8.4689999999999994,"5min":8.4689999999999994,"15min":8.4689999999999994},"last":0.315},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.23699999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47699999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.30399999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.27700000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.57399999999999995,"15min":0.51400000000000001},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":8.5299999999999994,"5min":8.5299999999999994,"15min":8.5299999999999994},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.503,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71199999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.496}]},{"osd":5,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.65100000000000002,"15min":0.60299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":8.6039999999999992,"5min":8.6039999999999992,"15min":8.6039999999999992},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.55300000000000005,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69099999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.39500000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.64200000000000002,"5min":1.075,"15min":1.075},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.67500000000000004,"15min":0.628},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":8.5809999999999995,"5min":8.5809999999999995,"15min":8.5809999999999995},"last":0.46800000000000003}]},{"osd":7,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64800000000000002,"5min":0.747,"15min":0.747},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.61199999999999999,"15min":0.55200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":8.5570000000000004,"5min":8.5570000000000004,"15min":8.5570000000000004},"last":0.42399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:45:59.752 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T11:45:59.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-16T11:46:00.033 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T11:46:00.046 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-16T11:46:00.046 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:46:00.305 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:46:00.305 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":147,"fsid":"3420bcad-625f-4e15-aaff-fb4236322168","created":"2024-09-16T11:43:01.654626+0000","modified":"2024-09-16T11:45:54.705133+0000","last_up_change":"2024-09-16T11:45:38.626538+0000","last_in_change":"2024-09-16T11:43:04.526236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:43:11.406582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:43:13.125709+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"23","last_force_op_resend_prenautilus":"23","last_force_op_resend_preluminous":"23","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T11:43:21.227888+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"23","last_force_op_resend_prenautilus":"23","last_force_op_resend_preluminous":"23","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T11:43:30.105533+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":83,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"91f052e4-4c8f-4484-92ce-bd1a7f8f3efb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":33355},{"type":"v1","addr":"172.21.15.70:6801","nonce":33355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":33355},{"type":"v1","addr":"172.21.15.70:6803","nonce":33355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":33355},{"type":"v1","addr":"172.21.15.70:6807","nonce":33355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":33355},{"type":"v1","addr":"172.21.15.70:6805","nonce":33355}]},"public_addr":"172.21.15.70:6801/33355","cluster_addr":"172.21.15.70:6803/33355","heartbeat_back_addr":"172.21.15.70:6807/33355","heartbeat_front_addr":"172.21.15.70:6805/33355","state":["exists","up"]},{"osd":1,"uuid":"568b1280-7365-4094-afe8-b0b5e766de6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":33359},{"type":"v1","addr":"172.21.15.70:6817","nonce":33359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":33359},{"type":"v1","addr":"172.21.15.70:6819","nonce":33359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":33359},{"type":"v1","addr":"172.21.15.70:6823","nonce":33359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":33359},{"type":"v1","addr":"172.21.15.70:6821","nonce":33359}]},"public_addr":"172.21.15.70:6817/33359","cluster_addr":"172.21.15.70:6819/33359","heartbeat_back_addr":"172.21.15.70:6823/33359","heartbeat_front_addr":"172.21.15.70:6821/33359","state":["exists","up"]},{"osd":2,"uuid":"328adfa3-c1b4-4745-9759-7327c8cbd499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":33356},{"type":"v1","addr":"172.21.15.70:6809","nonce":33356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":33356},{"type":"v1","addr":"172.21.15.70:6811","nonce":33356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":33356},{"type":"v1","addr":"172.21.15.70:6815","nonce":33356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":33356},{"type":"v1","addr":"172.21.15.70:6813","nonce":33356}]},"public_addr":"172.21.15.70:6809/33356","cluster_addr":"172.21.15.70:6811/33356","heartbeat_back_addr":"172.21.15.70:6815/33356","heartbeat_front_addr":"172.21.15.70:6813/33356","state":["exists","up"]},{"osd":3,"uuid":"6df66ae7-e835-4d8f-bd27-243d7a9705b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":112,"up_from":131,"up_thru":131,"down_at":113,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":58254},{"type":"v1","addr":"172.21.15.70:6825","nonce":58254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":58254},{"type":"v1","addr":"172.21.15.70:6827","nonce":58254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":58254},{"type":"v1","addr":"172.21.15.70:6831","nonce":58254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":58254},{"type":"v1","addr":"172.21.15.70:6829","nonce":58254}]},"public_addr":"172.21.15.70:6825/58254","cluster_addr":"172.21.15.70:6827/58254","heartbeat_back_addr":"172.21.15.70:6831/58254","heartbeat_front_addr":"172.21.15.70:6829/58254","state":["exists","up"]},{"osd":4,"uuid":"29f52d62-d4c3-49f9-992f-23817f5622d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":33331},{"type":"v1","addr":"172.21.15.192:6803","nonce":33331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":33331},{"type":"v1","addr":"172.21.15.192:6807","nonce":33331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6813","nonce":33331},{"type":"v1","addr":"172.21.15.192:6815","nonce":33331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":33331},{"type":"v1","addr":"172.21.15.192:6811","nonce":33331}]},"public_addr":"172.21.15.192:6803/33331","cluster_addr":"172.21.15.192:6807/33331","heartbeat_back_addr":"172.21.15.192:6815/33331","heartbeat_front_addr":"172.21.15.192:6811/33331","state":["exists","up"]},{"osd":5,"uuid":"0c1a275a-3c75-4e0b-bd35-d34b64d99c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":33333},{"type":"v1","addr":"172.21.15.192:6817","nonce":33333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":33333},{"type":"v1","addr":"172.21.15.192:6819","nonce":33333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":33333},{"type":"v1","addr":"172.21.15.192:6823","nonce":33333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":33333},{"type":"v1","addr":"172.21.15.192:6821","nonce":33333}]},"public_addr":"172.21.15.192:6817/33333","cluster_addr":"172.21.15.192:6819/33333","heartbeat_back_addr":"172.21.15.192:6823/33333","heartbeat_front_addr":"172.21.15.192:6821/33333","state":["exists","up"]},{"osd":6,"uuid":"d79f0070-ce58-48e7-a56a-2762ec76a271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6801","nonce":33332},{"type":"v1","addr":"172.21.15.192:6802","nonce":33332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":33332},{"type":"v1","addr":"172.21.15.192:6806","nonce":33332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":33332},{"type":"v1","addr":"172.21.15.192:6814","nonce":33332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":33332},{"type":"v1","addr":"172.21.15.192:6810","nonce":33332}]},"public_addr":"172.21.15.192:6802/33332","cluster_addr":"172.21.15.192:6806/33332","heartbeat_back_addr":"172.21.15.192:6814/33332","heartbeat_front_addr":"172.21.15.192:6810/33332","state":["exists","up"]},{"osd":7,"uuid":"537fa5df-3d0a-4dd4-8155-20ee94e41c82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":33362},{"type":"v1","addr":"172.21.15.192:6825","nonce":33362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":33362},{"type":"v1","addr":"172.21.15.192:6827","nonce":33362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":33362},{"type":"v1","addr":"172.21.15.192:6831","nonce":33362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":33362},{"type":"v1","addr":"172.21.15.192:6829","nonce":33362}]},"public_addr":"172.21.15.192:6825/33362","cluster_addr":"172.21.15.192:6827/33362","heartbeat_back_addr":"172.21.15.192:6831/33362","heartbeat_front_addr":"172.21.15.192:6829/33362","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.581514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.674271+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.545402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.733565+0000","dead_epoch":113},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.623479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.730066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.556650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.702368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":5,"to":1}]},{"pgid":"4.14","mappings":[{"from":1,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:46:00.317 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T11:46:00.317 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:46:00.578 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:46:00.578 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:46:00.589 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-16T11:45:59.402327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590574,"num_objects":741,"num_object_clones":0,"num_object_copies":1482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":731,"num_read":135541,"num_read_kb":642951,"num_write":18165,"num_write_kb":292238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16567,"ondisk_log_size":16567,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445748,"kb_used_data":13884,"kb_used_omap":54,"kb_used_meta":2431689,"kb_avail":752528972,"statfs":{"total":773094113280,"available":770589667328,"internally_reserved":0,"allocated":14217216,"data_stored":8312404,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56156,"internal_metadata":2490049700},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.705432"},"pg_stats":[{"pgid":"4.1f","version":"127'327","reported_seq":6237,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725885+0000","last_change":"2024-09-16T11:45:19.597844+0000","last_active":"2024-09-16T11:45:54.725885+0000","last_peered":"2024-09-16T11:45:54.725885+0000","last_clean":"2024-09-16T11:45:54.725885+0000","last_became_active":"2024-09-16T11:43:31.779428+0000","last_became_peered":"2024-09-16T11:43:31.779428+0000","last_unstale":"2024-09-16T11:45:54.725885+0000","last_undegraded":"2024-09-16T11:45:54.725885+0000","last_fullsized":"2024-09-16T11:45:54.725885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:30.437698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034041099999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6762,"num_read_kb":29123,"num_write":304,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"125'449","reported_seq":5346,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566266+0000","last_change":"2024-09-16T11:45:19.592718+0000","last_active":"2024-09-16T11:45:54.566266+0000","last_peered":"2024-09-16T11:45:54.566266+0000","last_clean":"2024-09-16T11:45:54.566266+0000","last_became_active":"2024-09-16T11:43:31.889706+0000","last_became_peered":"2024-09-16T11:43:31.889706+0000","last_unstale":"2024-09-16T11:45:54.566266+0000","last_undegraded":"2024-09-16T11:45:54.566266+0000","last_fullsized":"2024-09-16T11:45:54.566266+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:25.242116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000654113,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5672,"num_read_kb":36055,"num_write":486,"num_write_kb":9211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"132'213","reported_seq":1539,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725493+0000","last_change":"2024-09-16T11:45:39.661471+0000","last_active":"2024-09-16T11:45:54.725493+0000","last_peered":"2024-09-16T11:45:54.725493+0000","last_clean":"2024-09-16T11:45:54.725493+0000","last_became_active":"2024-09-16T11:45:39.636105+0000","last_became_peered":"2024-09-16T11:45:39.636105+0000","last_unstale":"2024-09-16T11:45:54.725493+0000","last_undegraded":"2024-09-16T11:45:54.725493+0000","last_fullsized":"2024-09-16T11:45:54.725493+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:18:52.260065+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.014051187999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1102,"num_read_kb":9776,"num_write":200,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"127'334","reported_seq":2413,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703281+0000","last_change":"2024-09-16T11:45:19.592554+0000","last_active":"2024-09-16T11:45:53.703281+0000","last_peered":"2024-09-16T11:45:53.703281+0000","last_clean":"2024-09-16T11:45:53.703281+0000","last_became_active":"2024-09-16T11:43:31.773661+0000","last_became_peered":"2024-09-16T11:43:31.773661+0000","last_unstale":"2024-09-16T11:45:53.703281+0000","last_undegraded":"2024-09-16T11:45:53.703281+0000","last_fullsized":"2024-09-16T11:45:53.703281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:36.062796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052992100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":2191,"num_read_kb":14785,"num_write":375,"num_write_kb":7446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"132'380","reported_seq":2524,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703902+0000","last_change":"2024-09-16T11:45:47.464749+0000","last_active":"2024-09-16T11:45:53.703902+0000","last_peered":"2024-09-16T11:45:53.703902+0000","last_clean":"2024-09-16T11:45:53.703902+0000","last_became_active":"2024-09-16T11:45:47.463209+0000","last_became_peered":"2024-09-16T11:45:47.463209+0000","last_unstale":"2024-09-16T11:45:53.703902+0000","last_undegraded":"2024-09-16T11:45:53.703902+0000","last_fullsized":"2024-09-16T11:45:53.703902+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:29.503448+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0033897049999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3246,"num_read_kb":18406,"num_write":318,"num_write_kb":4718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"132'534","reported_seq":3517,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703670+0000","last_change":"2024-09-16T11:45:47.464540+0000","last_active":"2024-09-16T11:45:53.703670+0000","last_peered":"2024-09-16T11:45:53.703670+0000","last_clean":"2024-09-16T11:45:53.703670+0000","last_became_active":"2024-09-16T11:45:47.463199+0000","last_became_peered":"2024-09-16T11:45:47.463199+0000","last_unstale":"2024-09-16T11:45:53.703670+0000","last_undegraded":"2024-09-16T11:45:53.703670+0000","last_fullsized":"2024-09-16T11:45:53.703670+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:44.772848+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.01209723,"stat_sum":{"num_bytes":19,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":4102,"num_read_kb":12532,"num_write":637,"num_write_kb":9975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"124'658","reported_seq":4607,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725383+0000","last_change":"2024-09-16T11:45:39.678403+0000","last_active":"2024-09-16T11:45:54.725383+0000","last_peered":"2024-09-16T11:45:54.725383+0000","last_clean":"2024-09-16T11:45:54.725383+0000","last_became_active":"2024-09-16T11:45:39.638104+0000","last_became_peered":"2024-09-16T11:45:39.638104+0000","last_unstale":"2024-09-16T11:45:54.725383+0000","last_undegraded":"2024-09-16T11:45:54.725383+0000","last_fullsized":"2024-09-16T11:45:54.725383+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:18.828819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069266390000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5632,"num_read_kb":29340,"num_write":868,"num_write_kb":4351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"127'221","reported_seq":1540,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703616+0000","last_change":"2024-09-16T11:45:19.592170+0000","last_active":"2024-09-16T11:45:53.703616+0000","last_peered":"2024-09-16T11:45:53.703616+0000","last_clean":"2024-09-16T11:45:53.703616+0000","last_became_active":"2024-09-16T11:43:31.776951+0000","last_became_peered":"2024-09-16T11:43:31.776951+0000","last_unstale":"2024-09-16T11:45:53.703616+0000","last_undegraded":"2024-09-16T11:45:53.703616+0000","last_fullsized":"2024-09-16T11:45:53.703616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:28:39.026621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052508100000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1522,"num_read_kb":7050,"num_write":185,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"132'525","reported_seq":2341,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725539+0000","last_change":"2024-09-16T11:45:39.723943+0000","last_active":"2024-09-16T11:45:54.725539+0000","last_peered":"2024-09-16T11:45:54.725539+0000","last_clean":"2024-09-16T11:45:54.725539+0000","last_became_active":"2024-09-16T11:45:39.642890+0000","last_became_peered":"2024-09-16T11:45:39.642890+0000","last_unstale":"2024-09-16T11:45:54.725539+0000","last_undegraded":"2024-09-16T11:45:54.725539+0000","last_fullsized":"2024-09-16T11:45:54.725539+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:26.709287+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062520060000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4283,"num_read_kb":14493,"num_write":541,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"132'743","reported_seq":6417,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566264+0000","last_change":"2024-09-16T11:45:39.656713+0000","last_active":"2024-09-16T11:45:54.566264+0000","last_peered":"2024-09-16T11:45:54.566264+0000","last_clean":"2024-09-16T11:45:54.566264+0000","last_became_active":"2024-09-16T11:45:39.640293+0000","last_became_peered":"2024-09-16T11:45:39.640293+0000","last_unstale":"2024-09-16T11:45:54.566264+0000","last_undegraded":"2024-09-16T11:45:54.566264+0000","last_fullsized":"2024-09-16T11:45:54.566264+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:29:01.940843+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074727580000000004,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8298,"num_read_kb":47234,"num_write":833,"num_write_kb":15319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"127'657","reported_seq":2814,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725859+0000","last_change":"2024-09-16T11:45:19.636993+0000","last_active":"2024-09-16T11:45:54.725859+0000","last_peered":"2024-09-16T11:45:54.725859+0000","last_clean":"2024-09-16T11:45:54.725859+0000","last_became_active":"2024-09-16T11:43:31.889629+0000","last_became_peered":"2024-09-16T11:43:31.889629+0000","last_unstale":"2024-09-16T11:45:54.725859+0000","last_undegraded":"2024-09-16T11:45:54.725859+0000","last_fullsized":"2024-09-16T11:45:54.725859+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:53.124562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030057499999999997,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":3982,"num_read_kb":26490,"num_write":811,"num_write_kb":16305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"132'665","reported_seq":3461,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725390+0000","last_change":"2024-09-16T11:45:39.707640+0000","last_active":"2024-09-16T11:45:54.725390+0000","last_peered":"2024-09-16T11:45:54.725390+0000","last_clean":"2024-09-16T11:45:54.725390+0000","last_became_active":"2024-09-16T11:45:39.641187+0000","last_became_peered":"2024-09-16T11:45:39.641187+0000","last_unstale":"2024-09-16T11:45:54.725390+0000","last_undegraded":"2024-09-16T11:45:54.725390+0000","last_fullsized":"2024-09-16T11:45:54.725390+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:47:39.464422+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074563379999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4363,"num_read_kb":20582,"num_write":914,"num_write_kb":10744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":3197941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"127'546","reported_seq":2511,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703790+0000","last_change":"2024-09-16T11:45:19.592153+0000","last_active":"2024-09-16T11:45:53.703790+0000","last_peered":"2024-09-16T11:45:53.703790+0000","last_clean":"2024-09-16T11:45:53.703790+0000","last_became_active":"2024-09-16T11:43:31.776781+0000","last_became_peered":"2024-09-16T11:43:31.776781+0000","last_unstale":"2024-09-16T11:45:53.703790+0000","last_undegraded":"2024-09-16T11:45:53.703790+0000","last_fullsized":"2024-09-16T11:45:53.703790+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:14.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032933499999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3911,"num_read_kb":15179,"num_write":595,"num_write_kb":10620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"121'294","reported_seq":1908,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703771+0000","last_change":"2024-09-16T11:45:19.642323+0000","last_active":"2024-09-16T11:45:53.703771+0000","last_peered":"2024-09-16T11:45:53.703771+0000","last_clean":"2024-09-16T11:45:53.703771+0000","last_became_active":"2024-09-16T11:43:31.780164+0000","last_became_peered":"2024-09-16T11:43:31.780164+0000","last_unstale":"2024-09-16T11:45:53.703771+0000","last_undegraded":"2024-09-16T11:45:53.703771+0000","last_fullsized":"2024-09-16T11:45:53.703771+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:56.846891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079981,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1687,"num_read_kb":10324,"num_write":299,"num_write_kb":6058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"132'804","reported_seq":2260,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725294+0000","last_change":"2024-09-16T11:45:39.712869+0000","last_active":"2024-09-16T11:45:54.725294+0000","last_peered":"2024-09-16T11:45:54.725294+0000","last_clean":"2024-09-16T11:45:54.725294+0000","last_became_active":"2024-09-16T11:45:39.641126+0000","last_became_peered":"2024-09-16T11:45:39.641126+0000","last_unstale":"2024-09-16T11:45:54.725294+0000","last_undegraded":"2024-09-16T11:45:54.725294+0000","last_fullsized":"2024-09-16T11:45:54.725294+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:23.732386+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074597470000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":4259,"num_read_kb":18254,"num_write":1055,"num_write_kb":13061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"132'401","reported_seq":1908,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725271+0000","last_change":"2024-09-16T11:45:39.666850+0000","last_active":"2024-09-16T11:45:54.725271+0000","last_peered":"2024-09-16T11:45:54.725271+0000","last_clean":"2024-09-16T11:45:54.725271+0000","last_became_active":"2024-09-16T11:45:39.637293+0000","last_became_peered":"2024-09-16T11:45:39.637293+0000","last_unstale":"2024-09-16T11:45:54.725271+0000","last_undegraded":"2024-09-16T11:45:54.725271+0000","last_fullsized":"2024-09-16T11:45:54.725271+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:39.210719+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.016205368000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3249,"num_read_kb":9637,"num_write":346,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725801+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:45:54.725801+0000","last_peered":"2024-09-16T11:45:54.725801+0000","last_clean":"2024-09-16T11:45:54.725801+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:45:54.725801+0000","last_undegraded":"2024-09-16T11:45:54.725801+0000","last_fullsized":"2024-09-16T11:45:54.725801+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"125'347","reported_seq":1761,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703661+0000","last_change":"2024-09-16T11:45:47.465149+0000","last_active":"2024-09-16T11:45:53.703661+0000","last_peered":"2024-09-16T11:45:53.703661+0000","last_clean":"2024-09-16T11:45:53.703661+0000","last_became_active":"2024-09-16T11:45:47.464115+0000","last_became_peered":"2024-09-16T11:45:47.464115+0000","last_unstale":"2024-09-16T11:45:53.703661+0000","last_undegraded":"2024-09-16T11:45:53.703661+0000","last_fullsized":"2024-09-16T11:45:53.703661+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:34.356992+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00038392500000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2491,"num_read_kb":9150,"num_write":283,"num_write_kb":9998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703962+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:45:53.703962+0000","last_peered":"2024-09-16T11:45:53.703962+0000","last_clean":"2024-09-16T11:45:53.703962+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:45:53.703962+0000","last_undegraded":"2024-09-16T11:45:53.703962+0000","last_fullsized":"2024-09-16T11:45:53.703962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"132'477","reported_seq":4581,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725271+0000","last_change":"2024-09-16T11:45:39.723814+0000","last_active":"2024-09-16T11:45:54.725271+0000","last_peered":"2024-09-16T11:45:54.725271+0000","last_clean":"2024-09-16T11:45:54.725271+0000","last_became_active":"2024-09-16T11:45:39.641298+0000","last_became_peered":"2024-09-16T11:45:39.641298+0000","last_unstale":"2024-09-16T11:45:54.725271+0000","last_undegraded":"2024-09-16T11:45:54.725271+0000","last_fullsized":"2024-09-16T11:45:54.725271+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:57.805852+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0094940819999999992,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5544,"num_read_kb":23694,"num_write":515,"num_write_kb":8223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.722985+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:45:53.722985+0000","last_peered":"2024-09-16T11:45:53.722985+0000","last_clean":"2024-09-16T11:45:53.722985+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:45:53.722985+0000","last_undegraded":"2024-09-16T11:45:53.722985+0000","last_fullsized":"2024-09-16T11:45:53.722985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"127'627","reported_seq":6867,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566281+0000","last_change":"2024-09-16T11:45:19.593742+0000","last_active":"2024-09-16T11:45:54.566281+0000","last_peered":"2024-09-16T11:45:54.566281+0000","last_clean":"2024-09-16T11:45:54.566281+0000","last_became_active":"2024-09-16T11:43:31.777896+0000","last_became_peered":"2024-09-16T11:43:31.777896+0000","last_unstale":"2024-09-16T11:45:54.566281+0000","last_undegraded":"2024-09-16T11:45:54.566281+0000","last_fullsized":"2024-09-16T11:45:54.566281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:35.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011770719999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":7410,"num_read_kb":47228,"num_write":617,"num_write_kb":8894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704146+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:45:53.704146+0000","last_peered":"2024-09-16T11:45:53.704146+0000","last_clean":"2024-09-16T11:45:53.704146+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:45:53.704146+0000","last_undegraded":"2024-09-16T11:45:53.704146+0000","last_fullsized":"2024-09-16T11:45:53.704146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"126'382","reported_seq":1083,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703876+0000","last_change":"2024-09-16T11:45:19.642384+0000","last_active":"2024-09-16T11:45:53.703876+0000","last_peered":"2024-09-16T11:45:53.703876+0000","last_clean":"2024-09-16T11:45:53.703876+0000","last_became_active":"2024-09-16T11:43:31.778506+0000","last_became_peered":"2024-09-16T11:43:31.778506+0000","last_unstale":"2024-09-16T11:45:53.703876+0000","last_undegraded":"2024-09-16T11:45:53.703876+0000","last_fullsized":"2024-09-16T11:45:53.703876+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:12.232493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011478079999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1438,"num_read_kb":4478,"num_write":491,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":110,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703900+0000","last_change":"2024-09-16T11:45:47.462953+0000","last_active":"2024-09-16T11:45:53.703900+0000","last_peered":"2024-09-16T11:45:53.703900+0000","last_clean":"2024-09-16T11:45:53.703900+0000","last_became_active":"2024-09-16T11:45:47.462295+0000","last_became_peered":"2024-09-16T11:45:47.462295+0000","last_unstale":"2024-09-16T11:45:53.703900+0000","last_undegraded":"2024-09-16T11:45:53.703900+0000","last_fullsized":"2024-09-16T11:45:53.703900+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:57.786498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":140,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725763+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:45:54.725763+0000","last_peered":"2024-09-16T11:45:54.725763+0000","last_clean":"2024-09-16T11:45:54.725763+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:45:54.725763+0000","last_undegraded":"2024-09-16T11:45:54.725763+0000","last_fullsized":"2024-09-16T11:45:54.725763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"127'734","reported_seq":2603,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704134+0000","last_change":"2024-09-16T11:45:19.641341+0000","last_active":"2024-09-16T11:45:53.704134+0000","last_peered":"2024-09-16T11:45:53.704134+0000","last_clean":"2024-09-16T11:45:53.704134+0000","last_became_active":"2024-09-16T11:43:31.777796+0000","last_became_peered":"2024-09-16T11:43:31.777796+0000","last_unstale":"2024-09-16T11:45:53.704134+0000","last_undegraded":"2024-09-16T11:45:53.704134+0000","last_fullsized":"2024-09-16T11:45:53.704134+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:33.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051098400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4214,"num_read_kb":15199,"num_write":913,"num_write_kb":10139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":132,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704039+0000","last_change":"2024-09-16T11:43:22.440136+0000","last_active":"2024-09-16T11:45:53.704039+0000","last_peered":"2024-09-16T11:45:53.704039+0000","last_clean":"2024-09-16T11:45:53.704039+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:45:53.704039+0000","last_undegraded":"2024-09-16T11:45:53.704039+0000","last_fullsized":"2024-09-16T11:45:53.704039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:18.461948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703187+0000","last_change":"2024-09-16T11:45:39.635580+0000","last_active":"2024-09-16T11:45:53.703187+0000","last_peered":"2024-09-16T11:45:53.703187+0000","last_clean":"2024-09-16T11:45:53.703187+0000","last_became_active":"2024-09-16T11:45:39.635334+0000","last_became_peered":"2024-09-16T11:45:39.635334+0000","last_unstale":"2024-09-16T11:45:53.703187+0000","last_undegraded":"2024-09-16T11:45:53.703187+0000","last_fullsized":"2024-09-16T11:45:53.703187+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:43.033024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"125'546","reported_seq":3390,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704084+0000","last_change":"2024-09-16T11:45:19.641909+0000","last_active":"2024-09-16T11:45:53.704084+0000","last_peered":"2024-09-16T11:45:53.704084+0000","last_clean":"2024-09-16T11:45:53.704084+0000","last_became_active":"2024-09-16T11:43:31.777967+0000","last_became_peered":"2024-09-16T11:43:31.777967+0000","last_unstale":"2024-09-16T11:45:53.704084+0000","last_undegraded":"2024-09-16T11:45:53.704084+0000","last_fullsized":"2024-09-16T11:45:53.704084+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:43.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081483900000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3645,"num_read_kb":23388,"num_write":630,"num_write_kb":9618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"112'2","reported_seq":133,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566166+0000","last_change":"2024-09-16T11:43:22.437965+0000","last_active":"2024-09-16T11:45:54.566166+0000","last_peered":"2024-09-16T11:45:54.566166+0000","last_clean":"2024-09-16T11:45:54.566166+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:45:54.566166+0000","last_undegraded":"2024-09-16T11:45:54.566166+0000","last_fullsized":"2024-09-16T11:45:54.566166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:52.884360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.723066+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:45:53.723066+0000","last_peered":"2024-09-16T11:45:53.723066+0000","last_clean":"2024-09-16T11:45:53.723066+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:45:53.723066+0000","last_undegraded":"2024-09-16T11:45:53.723066+0000","last_fullsized":"2024-09-16T11:45:53.723066+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"127'407","reported_seq":1770,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566334+0000","last_change":"2024-09-16T11:45:19.592986+0000","last_active":"2024-09-16T11:45:54.566334+0000","last_peered":"2024-09-16T11:45:54.566334+0000","last_clean":"2024-09-16T11:45:54.566334+0000","last_became_active":"2024-09-16T11:43:31.776645+0000","last_became_peered":"2024-09-16T11:43:31.776645+0000","last_unstale":"2024-09-16T11:45:54.566334+0000","last_undegraded":"2024-09-16T11:45:54.566334+0000","last_fullsized":"2024-09-16T11:45:54.566334+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:53.198323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066659399999999995,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1650,"num_read_kb":12114,"num_write":331,"num_write_kb":15245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"84'3","reported_seq":97,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725388+0000","last_change":"2024-09-16T11:45:39.638395+0000","last_active":"2024-09-16T11:45:54.725388+0000","last_peered":"2024-09-16T11:45:54.725388+0000","last_clean":"2024-09-16T11:45:54.725388+0000","last_became_active":"2024-09-16T11:45:39.637629+0000","last_became_peered":"2024-09-16T11:45:39.637629+0000","last_unstale":"2024-09-16T11:45:54.725388+0000","last_undegraded":"2024-09-16T11:45:54.725388+0000","last_fullsized":"2024-09-16T11:45:54.725388+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:56.174857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725989+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:45:54.725989+0000","last_peered":"2024-09-16T11:45:54.725989+0000","last_clean":"2024-09-16T11:45:54.725989+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:45:54.725989+0000","last_undegraded":"2024-09-16T11:45:54.725989+0000","last_fullsized":"2024-09-16T11:45:54.725989+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":226,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.723073+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:45:53.723073+0000","last_peered":"2024-09-16T11:45:53.723073+0000","last_clean":"2024-09-16T11:45:53.723073+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:45:53.723073+0000","last_undegraded":"2024-09-16T11:45:53.723073+0000","last_fullsized":"2024-09-16T11:45:53.723073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"125'523","reported_seq":3438,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725834+0000","last_change":"2024-09-16T11:45:19.598172+0000","last_active":"2024-09-16T11:45:54.725834+0000","last_peered":"2024-09-16T11:45:54.725834+0000","last_clean":"2024-09-16T11:45:54.725834+0000","last_became_active":"2024-09-16T11:43:31.782071+0000","last_became_peered":"2024-09-16T11:43:31.782071+0000","last_unstale":"2024-09-16T11:45:54.725834+0000","last_undegraded":"2024-09-16T11:45:54.725834+0000","last_fullsized":"2024-09-16T11:45:54.725834+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:15.516215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000599604,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3801,"num_read_kb":24361,"num_write":540,"num_write_kb":7978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"127'665","reported_seq":9006,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703769+0000","last_change":"2024-09-16T11:45:19.641414+0000","last_active":"2024-09-16T11:45:53.703769+0000","last_peered":"2024-09-16T11:45:53.703769+0000","last_clean":"2024-09-16T11:45:53.703769+0000","last_became_active":"2024-09-16T11:44:10.920131+0000","last_became_peered":"2024-09-16T11:44:10.920131+0000","last_unstale":"2024-09-16T11:45:53.703769+0000","last_undegraded":"2024-09-16T11:45:53.703769+0000","last_fullsized":"2024-09-16T11:45:53.703769+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:55:10.158414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082161099999999998,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":9498,"num_read_kb":69976,"num_write":712,"num_write_kb":19149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":8960357,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"127'878","reported_seq":3861,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703624+0000","last_change":"2024-09-16T11:45:47.465826+0000","last_active":"2024-09-16T11:45:53.703624+0000","last_peered":"2024-09-16T11:45:53.703624+0000","last_clean":"2024-09-16T11:45:53.703624+0000","last_became_active":"2024-09-16T11:45:47.464353+0000","last_became_peered":"2024-09-16T11:45:47.464353+0000","last_unstale":"2024-09-16T11:45:53.703624+0000","last_undegraded":"2024-09-16T11:45:53.703624+0000","last_fullsized":"2024-09-16T11:45:53.703624+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:24.438451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001068741,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":8005,"num_read_kb":12353,"num_write":894,"num_write_kb":6127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"127'623","reported_seq":1817,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.726002+0000","last_change":"2024-09-16T11:45:19.598161+0000","last_active":"2024-09-16T11:45:54.726002+0000","last_peered":"2024-09-16T11:45:54.726002+0000","last_clean":"2024-09-16T11:45:54.726002+0000","last_became_active":"2024-09-16T11:43:31.779384+0000","last_became_peered":"2024-09-16T11:43:31.779384+0000","last_unstale":"2024-09-16T11:45:54.726002+0000","last_undegraded":"2024-09-16T11:45:54.726002+0000","last_fullsized":"2024-09-16T11:45:54.726002+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:34.933146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000647592,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":2523,"num_read_kb":9199,"num_write":666,"num_write_kb":6203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"125'377","reported_seq":3380,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566193+0000","last_change":"2024-09-16T11:45:19.593453+0000","last_active":"2024-09-16T11:45:54.566193+0000","last_peered":"2024-09-16T11:45:54.566193+0000","last_clean":"2024-09-16T11:45:54.566193+0000","last_became_active":"2024-09-16T11:43:31.775335+0000","last_became_peered":"2024-09-16T11:43:31.775335+0000","last_unstale":"2024-09-16T11:45:54.566193+0000","last_undegraded":"2024-09-16T11:45:54.566193+0000","last_fullsized":"2024-09-16T11:45:54.566193+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:07.521713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011352389999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4122,"num_read_kb":15191,"num_write":417,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"132'521","reported_seq":2268,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703740+0000","last_change":"2024-09-16T11:45:47.465501+0000","last_active":"2024-09-16T11:45:53.703740+0000","last_peered":"2024-09-16T11:45:53.703740+0000","last_clean":"2024-09-16T11:45:53.703740+0000","last_became_active":"2024-09-16T11:45:47.463487+0000","last_became_peered":"2024-09-16T11:45:47.463487+0000","last_unstale":"2024-09-16T11:45:53.703740+0000","last_undegraded":"2024-09-16T11:45:53.703740+0000","last_fullsized":"2024-09-16T11:45:53.703740+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:41.597993+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001754511,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4037,"num_read_kb":5871,"num_write":454,"num_write_kb":2926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"125'616","reported_seq":5073,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703823+0000","last_change":"2024-09-16T11:45:47.462731+0000","last_active":"2024-09-16T11:45:53.703823+0000","last_peered":"2024-09-16T11:45:53.703823+0000","last_clean":"2024-09-16T11:45:53.703823+0000","last_became_active":"2024-09-16T11:45:47.461210+0000","last_became_peered":"2024-09-16T11:45:47.461210+0000","last_unstale":"2024-09-16T11:45:53.703823+0000","last_undegraded":"2024-09-16T11:45:53.703823+0000","last_fullsized":"2024-09-16T11:45:53.703823+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:17.838544+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00043040400000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6463,"num_read_kb":36386,"num_write":811,"num_write_kb":14056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"126'605","reported_seq":1931,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703756+0000","last_change":"2024-09-16T11:45:47.465926+0000","last_active":"2024-09-16T11:45:53.703756+0000","last_peered":"2024-09-16T11:45:53.703756+0000","last_clean":"2024-09-16T11:45:53.703756+0000","last_became_active":"2024-09-16T11:45:47.464052+0000","last_became_peered":"2024-09-16T11:45:47.464052+0000","last_unstale":"2024-09-16T11:45:53.703756+0000","last_undegraded":"2024-09-16T11:45:53.703756+0000","last_fullsized":"2024-09-16T11:45:53.703756+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:40.083173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059308799999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2764,"num_read_kb":9887,"num_write":657,"num_write_kb":5333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"127'408","reported_seq":2117,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703980+0000","last_change":"2024-09-16T11:45:39.667068+0000","last_active":"2024-09-16T11:45:53.703980+0000","last_peered":"2024-09-16T11:45:53.703980+0000","last_clean":"2024-09-16T11:45:53.703980+0000","last_became_active":"2024-09-16T11:45:39.643207+0000","last_became_peered":"2024-09-16T11:45:39.643207+0000","last_unstale":"2024-09-16T11:45:53.703980+0000","last_undegraded":"2024-09-16T11:45:53.703980+0000","last_fullsized":"2024-09-16T11:45:53.703980+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:12.700226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070987940000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3609,"num_read_kb":5161,"num_write":362,"num_write_kb":8941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":728,"num_read":135475,"num_read_kb":642896,"num_write":18060,"num_write_kb":290606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},"log_size":16487,"ondisk_log_size":16487,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299128,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072712,"statfs":{"total":96636764160,"available":96330457088,"internally_reserved":0,"allocated":1814528,"data_stored":1481880,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.441,"15min":0.443},"min":{"1min":0.19600000000000001,"5min":0.182,"15min":0.182},"max":{"1min":0.61299999999999999,"5min":0.628,"15min":0.628},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.66000000000000003}]},{"osd":1,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.247,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":4.3559999999999999,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.435,"15min":0.435},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.67900000000000005}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.68400000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.623},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":1.532,"15min":1.532},"last":0.60799999999999998}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.5640000000000001,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.26000000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.67500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45400000000000001,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.437,"15min":0.44},"min":{"1min":0.27600000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65200000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.441},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.318,"5min":0.251,"15min":0.251},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.54700000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.377,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.5},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66200000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52900000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315728,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94056112,"statfs":{"total":96636764160,"available":96313458688,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.438,"15min":0.44400000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52800000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58999999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.503}]},{"osd":1,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.33800000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.53100000000000003,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.439,"15min":0.442},"min":{"1min":0.27400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.61799999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.30399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.35899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.63,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.621}]},{"osd":3,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.55800000000000005}]},{"osd":4,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.64700000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64300000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.48699999999999999,"15min":0.498},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.502,"15min":0.51100000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59899999999999998}]}]},{"osd":3,"up_from":131,"seq":562640715782,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302168,"kb_used_data":1728,"kb_used_omap":43,"kb_used_meta":300372,"kb_avail":94069672,"statfs":{"total":96636764160,"available":96327344128,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44228,"internal_metadata":307581756},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315724,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94056116,"statfs":{"total":96636764160,"available":96313462784,"internally_reserved":0,"allocated":1765376,"data_stored":892439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":1.087,"5min":0.495,"15min":0.39700000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":11.218,"5min":11.218,"15min":11.218},"last":0.31}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32200000000000001},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.372,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.36099999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.38100000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72399999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53500000000000003,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66600000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68600000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53900000000000003}]},{"osd":7,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.871,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311540,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94060300,"statfs":{"total":96636764160,"available":96317747200,"internally_reserved":0,"allocated":1744896,"data_stored":885224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.22700000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.59899999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.379},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.42999999999999999,"15min":0.438},"min":{"1min":0.22900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.61099999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.53700000000000003}]},{"osd":1,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.45300000000000001,"15min":0.45700000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.75800000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47599999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48399999999999999,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":1.026,"15min":1.026},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.47499999999999998,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79600000000000004,"5min":1.01,"15min":1.01},"last":0.56100000000000005}]},{"osd":3,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.622}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39800000000000002,"15min":0.40300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54700000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.502},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.27300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.69899999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.45300000000000001,"15min":0.46500000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.755,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.42099999999999999,"15min":0.433},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52400000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.48199999999999998,"15min":0.495},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67100000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303244,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068596,"statfs":{"total":96636764160,"available":96326242304,"internally_reserved":0,"allocated":1769472,"data_stored":892522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.47599999999999998,"15min":0.441},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":5.8399999999999999,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.44900000000000001,"15min":0.45100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.58699999999999997}]},{"osd":1,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.60699999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.48599999999999999,"15min":0.496},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.621,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.435,"5min":0.49099999999999999,"15min":0.5},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.73499999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.70099999999999996}]},{"osd":5,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.42099999999999999,"15min":0.435},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.44700000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.45500000000000002,"15min":0.46899999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.76400000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.505,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.53500000000000003,"15min":0.55100000000000005},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60299999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.81699999999999995}]},{"osd":7,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56599999999999995,"5min":0.873,"15min":0.873},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58999999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299076,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072764,"statfs":{"total":96636764160,"available":96330510336,"internally_reserved":0,"allocated":1761280,"data_stored":892420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.26200000000000001,"5min":0.29399999999999998,"15min":0.29999999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.46500000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.32800000000000001,"15min":0.32500000000000001},"min":{"1min":0.215,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.255}]},{"osd":1,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32900000000000001,"15min":0.33200000000000002},"min":{"1min":0.23000000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.437,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34799999999999998,"15min":0.34899999999999998},"min":{"1min":0.255,"5min":0.192,"15min":0.192},"max":{"1min":0.53600000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.40400000000000003}]},{"osd":4,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.52700000000000002,"15min":0.54100000000000004},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.35599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.216}]},{"osd":5,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":1.254}]},{"osd":6,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57999999999999996,"15min":0.58899999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":1.288},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.57799999999999996,"15min":0.58799999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.3089999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.266}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299140,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072700,"statfs":{"total":96636764160,"available":96330444800,"internally_reserved":0,"allocated":1822720,"data_stored":1482871,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.39600000000000002,"15min":0.33800000000000002},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":8.5050000000000008,"5min":8.5050000000000008,"15min":8.5050000000000008},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32100000000000001},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.436,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.26200000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.39600000000000002,"15min":0.33600000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":8.4689999999999994,"5min":8.4689999999999994,"15min":8.4689999999999994},"last":0.315},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.23699999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47699999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.30399999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.27700000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.57399999999999995,"15min":0.51400000000000001},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":8.5299999999999994,"5min":8.5299999999999994,"15min":8.5299999999999994},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.503,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71199999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.496}]},{"osd":5,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.65100000000000002,"15min":0.60299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":8.6039999999999992,"5min":8.6039999999999992,"15min":8.6039999999999992},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.55300000000000005,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69099999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.39500000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.64200000000000002,"5min":1.075,"15min":1.075},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.67500000000000004,"15min":0.628},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":8.5809999999999995,"5min":8.5809999999999995,"15min":8.5809999999999995},"last":0.46800000000000003}]},{"osd":7,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64800000000000002,"5min":0.747,"15min":0.747},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.61199999999999999,"15min":0.55200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":8.5570000000000004,"5min":8.5570000000000004,"15min":8.5570000000000004},"last":0.42399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:46:00.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:46:00.853 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:46:00.854 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:46:00.864 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-16T11:45:59.402327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590574,"num_objects":741,"num_object_clones":0,"num_object_copies":1482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":731,"num_read":135541,"num_read_kb":642951,"num_write":18165,"num_write_kb":292238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16567,"ondisk_log_size":16567,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445748,"kb_used_data":13884,"kb_used_omap":54,"kb_used_meta":2431689,"kb_avail":752528972,"statfs":{"total":773094113280,"available":770589667328,"internally_reserved":0,"allocated":14217216,"data_stored":8312404,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56156,"internal_metadata":2490049700},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.705432"},"pg_stats":[{"pgid":"4.1f","version":"127'327","reported_seq":6237,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725885+0000","last_change":"2024-09-16T11:45:19.597844+0000","last_active":"2024-09-16T11:45:54.725885+0000","last_peered":"2024-09-16T11:45:54.725885+0000","last_clean":"2024-09-16T11:45:54.725885+0000","last_became_active":"2024-09-16T11:43:31.779428+0000","last_became_peered":"2024-09-16T11:43:31.779428+0000","last_unstale":"2024-09-16T11:45:54.725885+0000","last_undegraded":"2024-09-16T11:45:54.725885+0000","last_fullsized":"2024-09-16T11:45:54.725885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:30.437698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034041099999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6762,"num_read_kb":29123,"num_write":304,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"125'449","reported_seq":5346,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566266+0000","last_change":"2024-09-16T11:45:19.592718+0000","last_active":"2024-09-16T11:45:54.566266+0000","last_peered":"2024-09-16T11:45:54.566266+0000","last_clean":"2024-09-16T11:45:54.566266+0000","last_became_active":"2024-09-16T11:43:31.889706+0000","last_became_peered":"2024-09-16T11:43:31.889706+0000","last_unstale":"2024-09-16T11:45:54.566266+0000","last_undegraded":"2024-09-16T11:45:54.566266+0000","last_fullsized":"2024-09-16T11:45:54.566266+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:25.242116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000654113,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5672,"num_read_kb":36055,"num_write":486,"num_write_kb":9211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"132'213","reported_seq":1539,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725493+0000","last_change":"2024-09-16T11:45:39.661471+0000","last_active":"2024-09-16T11:45:54.725493+0000","last_peered":"2024-09-16T11:45:54.725493+0000","last_clean":"2024-09-16T11:45:54.725493+0000","last_became_active":"2024-09-16T11:45:39.636105+0000","last_became_peered":"2024-09-16T11:45:39.636105+0000","last_unstale":"2024-09-16T11:45:54.725493+0000","last_undegraded":"2024-09-16T11:45:54.725493+0000","last_fullsized":"2024-09-16T11:45:54.725493+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:18:52.260065+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.014051187999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1102,"num_read_kb":9776,"num_write":200,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"127'334","reported_seq":2413,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703281+0000","last_change":"2024-09-16T11:45:19.592554+0000","last_active":"2024-09-16T11:45:53.703281+0000","last_peered":"2024-09-16T11:45:53.703281+0000","last_clean":"2024-09-16T11:45:53.703281+0000","last_became_active":"2024-09-16T11:43:31.773661+0000","last_became_peered":"2024-09-16T11:43:31.773661+0000","last_unstale":"2024-09-16T11:45:53.703281+0000","last_undegraded":"2024-09-16T11:45:53.703281+0000","last_fullsized":"2024-09-16T11:45:53.703281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:36.062796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052992100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":2191,"num_read_kb":14785,"num_write":375,"num_write_kb":7446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"132'380","reported_seq":2524,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703902+0000","last_change":"2024-09-16T11:45:47.464749+0000","last_active":"2024-09-16T11:45:53.703902+0000","last_peered":"2024-09-16T11:45:53.703902+0000","last_clean":"2024-09-16T11:45:53.703902+0000","last_became_active":"2024-09-16T11:45:47.463209+0000","last_became_peered":"2024-09-16T11:45:47.463209+0000","last_unstale":"2024-09-16T11:45:53.703902+0000","last_undegraded":"2024-09-16T11:45:53.703902+0000","last_fullsized":"2024-09-16T11:45:53.703902+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:29.503448+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0033897049999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3246,"num_read_kb":18406,"num_write":318,"num_write_kb":4718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"132'534","reported_seq":3517,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703670+0000","last_change":"2024-09-16T11:45:47.464540+0000","last_active":"2024-09-16T11:45:53.703670+0000","last_peered":"2024-09-16T11:45:53.703670+0000","last_clean":"2024-09-16T11:45:53.703670+0000","last_became_active":"2024-09-16T11:45:47.463199+0000","last_became_peered":"2024-09-16T11:45:47.463199+0000","last_unstale":"2024-09-16T11:45:53.703670+0000","last_undegraded":"2024-09-16T11:45:53.703670+0000","last_fullsized":"2024-09-16T11:45:53.703670+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:44.772848+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.01209723,"stat_sum":{"num_bytes":19,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":4102,"num_read_kb":12532,"num_write":637,"num_write_kb":9975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"124'658","reported_seq":4607,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725383+0000","last_change":"2024-09-16T11:45:39.678403+0000","last_active":"2024-09-16T11:45:54.725383+0000","last_peered":"2024-09-16T11:45:54.725383+0000","last_clean":"2024-09-16T11:45:54.725383+0000","last_became_active":"2024-09-16T11:45:39.638104+0000","last_became_peered":"2024-09-16T11:45:39.638104+0000","last_unstale":"2024-09-16T11:45:54.725383+0000","last_undegraded":"2024-09-16T11:45:54.725383+0000","last_fullsized":"2024-09-16T11:45:54.725383+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:18.828819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069266390000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5632,"num_read_kb":29340,"num_write":868,"num_write_kb":4351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"127'221","reported_seq":1540,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703616+0000","last_change":"2024-09-16T11:45:19.592170+0000","last_active":"2024-09-16T11:45:53.703616+0000","last_peered":"2024-09-16T11:45:53.703616+0000","last_clean":"2024-09-16T11:45:53.703616+0000","last_became_active":"2024-09-16T11:43:31.776951+0000","last_became_peered":"2024-09-16T11:43:31.776951+0000","last_unstale":"2024-09-16T11:45:53.703616+0000","last_undegraded":"2024-09-16T11:45:53.703616+0000","last_fullsized":"2024-09-16T11:45:53.703616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:28:39.026621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052508100000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1522,"num_read_kb":7050,"num_write":185,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"132'525","reported_seq":2341,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725539+0000","last_change":"2024-09-16T11:45:39.723943+0000","last_active":"2024-09-16T11:45:54.725539+0000","last_peered":"2024-09-16T11:45:54.725539+0000","last_clean":"2024-09-16T11:45:54.725539+0000","last_became_active":"2024-09-16T11:45:39.642890+0000","last_became_peered":"2024-09-16T11:45:39.642890+0000","last_unstale":"2024-09-16T11:45:54.725539+0000","last_undegraded":"2024-09-16T11:45:54.725539+0000","last_fullsized":"2024-09-16T11:45:54.725539+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:26.709287+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062520060000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4283,"num_read_kb":14493,"num_write":541,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"132'743","reported_seq":6417,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566264+0000","last_change":"2024-09-16T11:45:39.656713+0000","last_active":"2024-09-16T11:45:54.566264+0000","last_peered":"2024-09-16T11:45:54.566264+0000","last_clean":"2024-09-16T11:45:54.566264+0000","last_became_active":"2024-09-16T11:45:39.640293+0000","last_became_peered":"2024-09-16T11:45:39.640293+0000","last_unstale":"2024-09-16T11:45:54.566264+0000","last_undegraded":"2024-09-16T11:45:54.566264+0000","last_fullsized":"2024-09-16T11:45:54.566264+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:29:01.940843+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074727580000000004,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8298,"num_read_kb":47234,"num_write":833,"num_write_kb":15319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"127'657","reported_seq":2814,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725859+0000","last_change":"2024-09-16T11:45:19.636993+0000","last_active":"2024-09-16T11:45:54.725859+0000","last_peered":"2024-09-16T11:45:54.725859+0000","last_clean":"2024-09-16T11:45:54.725859+0000","last_became_active":"2024-09-16T11:43:31.889629+0000","last_became_peered":"2024-09-16T11:43:31.889629+0000","last_unstale":"2024-09-16T11:45:54.725859+0000","last_undegraded":"2024-09-16T11:45:54.725859+0000","last_fullsized":"2024-09-16T11:45:54.725859+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:53.124562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030057499999999997,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":3982,"num_read_kb":26490,"num_write":811,"num_write_kb":16305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"132'665","reported_seq":3461,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725390+0000","last_change":"2024-09-16T11:45:39.707640+0000","last_active":"2024-09-16T11:45:54.725390+0000","last_peered":"2024-09-16T11:45:54.725390+0000","last_clean":"2024-09-16T11:45:54.725390+0000","last_became_active":"2024-09-16T11:45:39.641187+0000","last_became_peered":"2024-09-16T11:45:39.641187+0000","last_unstale":"2024-09-16T11:45:54.725390+0000","last_undegraded":"2024-09-16T11:45:54.725390+0000","last_fullsized":"2024-09-16T11:45:54.725390+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:47:39.464422+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074563379999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4363,"num_read_kb":20582,"num_write":914,"num_write_kb":10744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":3197941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"127'546","reported_seq":2511,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703790+0000","last_change":"2024-09-16T11:45:19.592153+0000","last_active":"2024-09-16T11:45:53.703790+0000","last_peered":"2024-09-16T11:45:53.703790+0000","last_clean":"2024-09-16T11:45:53.703790+0000","last_became_active":"2024-09-16T11:43:31.776781+0000","last_became_peered":"2024-09-16T11:43:31.776781+0000","last_unstale":"2024-09-16T11:45:53.703790+0000","last_undegraded":"2024-09-16T11:45:53.703790+0000","last_fullsized":"2024-09-16T11:45:53.703790+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:14.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032933499999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3911,"num_read_kb":15179,"num_write":595,"num_write_kb":10620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"121'294","reported_seq":1908,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703771+0000","last_change":"2024-09-16T11:45:19.642323+0000","last_active":"2024-09-16T11:45:53.703771+0000","last_peered":"2024-09-16T11:45:53.703771+0000","last_clean":"2024-09-16T11:45:53.703771+0000","last_became_active":"2024-09-16T11:43:31.780164+0000","last_became_peered":"2024-09-16T11:43:31.780164+0000","last_unstale":"2024-09-16T11:45:53.703771+0000","last_undegraded":"2024-09-16T11:45:53.703771+0000","last_fullsized":"2024-09-16T11:45:53.703771+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:56.846891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079981,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1687,"num_read_kb":10324,"num_write":299,"num_write_kb":6058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"132'804","reported_seq":2260,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725294+0000","last_change":"2024-09-16T11:45:39.712869+0000","last_active":"2024-09-16T11:45:54.725294+0000","last_peered":"2024-09-16T11:45:54.725294+0000","last_clean":"2024-09-16T11:45:54.725294+0000","last_became_active":"2024-09-16T11:45:39.641126+0000","last_became_peered":"2024-09-16T11:45:39.641126+0000","last_unstale":"2024-09-16T11:45:54.725294+0000","last_undegraded":"2024-09-16T11:45:54.725294+0000","last_fullsized":"2024-09-16T11:45:54.725294+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:23.732386+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074597470000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":4259,"num_read_kb":18254,"num_write":1055,"num_write_kb":13061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"132'401","reported_seq":1908,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725271+0000","last_change":"2024-09-16T11:45:39.666850+0000","last_active":"2024-09-16T11:45:54.725271+0000","last_peered":"2024-09-16T11:45:54.725271+0000","last_clean":"2024-09-16T11:45:54.725271+0000","last_became_active":"2024-09-16T11:45:39.637293+0000","last_became_peered":"2024-09-16T11:45:39.637293+0000","last_unstale":"2024-09-16T11:45:54.725271+0000","last_undegraded":"2024-09-16T11:45:54.725271+0000","last_fullsized":"2024-09-16T11:45:54.725271+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:39.210719+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.016205368000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3249,"num_read_kb":9637,"num_write":346,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725801+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:45:54.725801+0000","last_peered":"2024-09-16T11:45:54.725801+0000","last_clean":"2024-09-16T11:45:54.725801+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:45:54.725801+0000","last_undegraded":"2024-09-16T11:45:54.725801+0000","last_fullsized":"2024-09-16T11:45:54.725801+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"125'347","reported_seq":1761,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703661+0000","last_change":"2024-09-16T11:45:47.465149+0000","last_active":"2024-09-16T11:45:53.703661+0000","last_peered":"2024-09-16T11:45:53.703661+0000","last_clean":"2024-09-16T11:45:53.703661+0000","last_became_active":"2024-09-16T11:45:47.464115+0000","last_became_peered":"2024-09-16T11:45:47.464115+0000","last_unstale":"2024-09-16T11:45:53.703661+0000","last_undegraded":"2024-09-16T11:45:53.703661+0000","last_fullsized":"2024-09-16T11:45:53.703661+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:34.356992+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00038392500000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2491,"num_read_kb":9150,"num_write":283,"num_write_kb":9998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703962+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:45:53.703962+0000","last_peered":"2024-09-16T11:45:53.703962+0000","last_clean":"2024-09-16T11:45:53.703962+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:45:53.703962+0000","last_undegraded":"2024-09-16T11:45:53.703962+0000","last_fullsized":"2024-09-16T11:45:53.703962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"132'477","reported_seq":4581,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725271+0000","last_change":"2024-09-16T11:45:39.723814+0000","last_active":"2024-09-16T11:45:54.725271+0000","last_peered":"2024-09-16T11:45:54.725271+0000","last_clean":"2024-09-16T11:45:54.725271+0000","last_became_active":"2024-09-16T11:45:39.641298+0000","last_became_peered":"2024-09-16T11:45:39.641298+0000","last_unstale":"2024-09-16T11:45:54.725271+0000","last_undegraded":"2024-09-16T11:45:54.725271+0000","last_fullsized":"2024-09-16T11:45:54.725271+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:57.805852+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0094940819999999992,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5544,"num_read_kb":23694,"num_write":515,"num_write_kb":8223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.722985+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:45:53.722985+0000","last_peered":"2024-09-16T11:45:53.722985+0000","last_clean":"2024-09-16T11:45:53.722985+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:45:53.722985+0000","last_undegraded":"2024-09-16T11:45:53.722985+0000","last_fullsized":"2024-09-16T11:45:53.722985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"127'627","reported_seq":6867,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566281+0000","last_change":"2024-09-16T11:45:19.593742+0000","last_active":"2024-09-16T11:45:54.566281+0000","last_peered":"2024-09-16T11:45:54.566281+0000","last_clean":"2024-09-16T11:45:54.566281+0000","last_became_active":"2024-09-16T11:43:31.777896+0000","last_became_peered":"2024-09-16T11:43:31.777896+0000","last_unstale":"2024-09-16T11:45:54.566281+0000","last_undegraded":"2024-09-16T11:45:54.566281+0000","last_fullsized":"2024-09-16T11:45:54.566281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:35.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011770719999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":7410,"num_read_kb":47228,"num_write":617,"num_write_kb":8894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704146+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:45:53.704146+0000","last_peered":"2024-09-16T11:45:53.704146+0000","last_clean":"2024-09-16T11:45:53.704146+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:45:53.704146+0000","last_undegraded":"2024-09-16T11:45:53.704146+0000","last_fullsized":"2024-09-16T11:45:53.704146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"126'382","reported_seq":1083,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703876+0000","last_change":"2024-09-16T11:45:19.642384+0000","last_active":"2024-09-16T11:45:53.703876+0000","last_peered":"2024-09-16T11:45:53.703876+0000","last_clean":"2024-09-16T11:45:53.703876+0000","last_became_active":"2024-09-16T11:43:31.778506+0000","last_became_peered":"2024-09-16T11:43:31.778506+0000","last_unstale":"2024-09-16T11:45:53.703876+0000","last_undegraded":"2024-09-16T11:45:53.703876+0000","last_fullsized":"2024-09-16T11:45:53.703876+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:12.232493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011478079999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1438,"num_read_kb":4478,"num_write":491,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":110,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703900+0000","last_change":"2024-09-16T11:45:47.462953+0000","last_active":"2024-09-16T11:45:53.703900+0000","last_peered":"2024-09-16T11:45:53.703900+0000","last_clean":"2024-09-16T11:45:53.703900+0000","last_became_active":"2024-09-16T11:45:47.462295+0000","last_became_peered":"2024-09-16T11:45:47.462295+0000","last_unstale":"2024-09-16T11:45:53.703900+0000","last_undegraded":"2024-09-16T11:45:53.703900+0000","last_fullsized":"2024-09-16T11:45:53.703900+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:57.786498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":140,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725763+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:45:54.725763+0000","last_peered":"2024-09-16T11:45:54.725763+0000","last_clean":"2024-09-16T11:45:54.725763+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:45:54.725763+0000","last_undegraded":"2024-09-16T11:45:54.725763+0000","last_fullsized":"2024-09-16T11:45:54.725763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"127'734","reported_seq":2603,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704134+0000","last_change":"2024-09-16T11:45:19.641341+0000","last_active":"2024-09-16T11:45:53.704134+0000","last_peered":"2024-09-16T11:45:53.704134+0000","last_clean":"2024-09-16T11:45:53.704134+0000","last_became_active":"2024-09-16T11:43:31.777796+0000","last_became_peered":"2024-09-16T11:43:31.777796+0000","last_unstale":"2024-09-16T11:45:53.704134+0000","last_undegraded":"2024-09-16T11:45:53.704134+0000","last_fullsized":"2024-09-16T11:45:53.704134+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:33.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051098400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4214,"num_read_kb":15199,"num_write":913,"num_write_kb":10139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":132,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704039+0000","last_change":"2024-09-16T11:43:22.440136+0000","last_active":"2024-09-16T11:45:53.704039+0000","last_peered":"2024-09-16T11:45:53.704039+0000","last_clean":"2024-09-16T11:45:53.704039+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:45:53.704039+0000","last_undegraded":"2024-09-16T11:45:53.704039+0000","last_fullsized":"2024-09-16T11:45:53.704039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:18.461948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703187+0000","last_change":"2024-09-16T11:45:39.635580+0000","last_active":"2024-09-16T11:45:53.703187+0000","last_peered":"2024-09-16T11:45:53.703187+0000","last_clean":"2024-09-16T11:45:53.703187+0000","last_became_active":"2024-09-16T11:45:39.635334+0000","last_became_peered":"2024-09-16T11:45:39.635334+0000","last_unstale":"2024-09-16T11:45:53.703187+0000","last_undegraded":"2024-09-16T11:45:53.703187+0000","last_fullsized":"2024-09-16T11:45:53.703187+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:43.033024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"125'546","reported_seq":3390,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704084+0000","last_change":"2024-09-16T11:45:19.641909+0000","last_active":"2024-09-16T11:45:53.704084+0000","last_peered":"2024-09-16T11:45:53.704084+0000","last_clean":"2024-09-16T11:45:53.704084+0000","last_became_active":"2024-09-16T11:43:31.777967+0000","last_became_peered":"2024-09-16T11:43:31.777967+0000","last_unstale":"2024-09-16T11:45:53.704084+0000","last_undegraded":"2024-09-16T11:45:53.704084+0000","last_fullsized":"2024-09-16T11:45:53.704084+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:43.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081483900000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3645,"num_read_kb":23388,"num_write":630,"num_write_kb":9618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"112'2","reported_seq":133,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566166+0000","last_change":"2024-09-16T11:43:22.437965+0000","last_active":"2024-09-16T11:45:54.566166+0000","last_peered":"2024-09-16T11:45:54.566166+0000","last_clean":"2024-09-16T11:45:54.566166+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:45:54.566166+0000","last_undegraded":"2024-09-16T11:45:54.566166+0000","last_fullsized":"2024-09-16T11:45:54.566166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:52.884360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.723066+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:45:53.723066+0000","last_peered":"2024-09-16T11:45:53.723066+0000","last_clean":"2024-09-16T11:45:53.723066+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:45:53.723066+0000","last_undegraded":"2024-09-16T11:45:53.723066+0000","last_fullsized":"2024-09-16T11:45:53.723066+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"127'407","reported_seq":1770,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566334+0000","last_change":"2024-09-16T11:45:19.592986+0000","last_active":"2024-09-16T11:45:54.566334+0000","last_peered":"2024-09-16T11:45:54.566334+0000","last_clean":"2024-09-16T11:45:54.566334+0000","last_became_active":"2024-09-16T11:43:31.776645+0000","last_became_peered":"2024-09-16T11:43:31.776645+0000","last_unstale":"2024-09-16T11:45:54.566334+0000","last_undegraded":"2024-09-16T11:45:54.566334+0000","last_fullsized":"2024-09-16T11:45:54.566334+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:53.198323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066659399999999995,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1650,"num_read_kb":12114,"num_write":331,"num_write_kb":15245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"84'3","reported_seq":97,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725388+0000","last_change":"2024-09-16T11:45:39.638395+0000","last_active":"2024-09-16T11:45:54.725388+0000","last_peered":"2024-09-16T11:45:54.725388+0000","last_clean":"2024-09-16T11:45:54.725388+0000","last_became_active":"2024-09-16T11:45:39.637629+0000","last_became_peered":"2024-09-16T11:45:39.637629+0000","last_unstale":"2024-09-16T11:45:54.725388+0000","last_undegraded":"2024-09-16T11:45:54.725388+0000","last_fullsized":"2024-09-16T11:45:54.725388+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:56.174857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725989+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:45:54.725989+0000","last_peered":"2024-09-16T11:45:54.725989+0000","last_clean":"2024-09-16T11:45:54.725989+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:45:54.725989+0000","last_undegraded":"2024-09-16T11:45:54.725989+0000","last_fullsized":"2024-09-16T11:45:54.725989+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":226,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.723073+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:45:53.723073+0000","last_peered":"2024-09-16T11:45:53.723073+0000","last_clean":"2024-09-16T11:45:53.723073+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:45:53.723073+0000","last_undegraded":"2024-09-16T11:45:53.723073+0000","last_fullsized":"2024-09-16T11:45:53.723073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"125'523","reported_seq":3438,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725834+0000","last_change":"2024-09-16T11:45:19.598172+0000","last_active":"2024-09-16T11:45:54.725834+0000","last_peered":"2024-09-16T11:45:54.725834+0000","last_clean":"2024-09-16T11:45:54.725834+0000","last_became_active":"2024-09-16T11:43:31.782071+0000","last_became_peered":"2024-09-16T11:43:31.782071+0000","last_unstale":"2024-09-16T11:45:54.725834+0000","last_undegraded":"2024-09-16T11:45:54.725834+0000","last_fullsized":"2024-09-16T11:45:54.725834+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:15.516215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000599604,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3801,"num_read_kb":24361,"num_write":540,"num_write_kb":7978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"127'665","reported_seq":9006,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703769+0000","last_change":"2024-09-16T11:45:19.641414+0000","last_active":"2024-09-16T11:45:53.703769+0000","last_peered":"2024-09-16T11:45:53.703769+0000","last_clean":"2024-09-16T11:45:53.703769+0000","last_became_active":"2024-09-16T11:44:10.920131+0000","last_became_peered":"2024-09-16T11:44:10.920131+0000","last_unstale":"2024-09-16T11:45:53.703769+0000","last_undegraded":"2024-09-16T11:45:53.703769+0000","last_fullsized":"2024-09-16T11:45:53.703769+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:55:10.158414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082161099999999998,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":9498,"num_read_kb":69976,"num_write":712,"num_write_kb":19149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":8960357,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"127'878","reported_seq":3861,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703624+0000","last_change":"2024-09-16T11:45:47.465826+0000","last_active":"2024-09-16T11:45:53.703624+0000","last_peered":"2024-09-16T11:45:53.703624+0000","last_clean":"2024-09-16T11:45:53.703624+0000","last_became_active":"2024-09-16T11:45:47.464353+0000","last_became_peered":"2024-09-16T11:45:47.464353+0000","last_unstale":"2024-09-16T11:45:53.703624+0000","last_undegraded":"2024-09-16T11:45:53.703624+0000","last_fullsized":"2024-09-16T11:45:53.703624+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:24.438451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001068741,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":8005,"num_read_kb":12353,"num_write":894,"num_write_kb":6127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"127'623","reported_seq":1817,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.726002+0000","last_change":"2024-09-16T11:45:19.598161+0000","last_active":"2024-09-16T11:45:54.726002+0000","last_peered":"2024-09-16T11:45:54.726002+0000","last_clean":"2024-09-16T11:45:54.726002+0000","last_became_active":"2024-09-16T11:43:31.779384+0000","last_became_peered":"2024-09-16T11:43:31.779384+0000","last_unstale":"2024-09-16T11:45:54.726002+0000","last_undegraded":"2024-09-16T11:45:54.726002+0000","last_fullsized":"2024-09-16T11:45:54.726002+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:34.933146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000647592,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":2523,"num_read_kb":9199,"num_write":666,"num_write_kb":6203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"125'377","reported_seq":3380,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566193+0000","last_change":"2024-09-16T11:45:19.593453+0000","last_active":"2024-09-16T11:45:54.566193+0000","last_peered":"2024-09-16T11:45:54.566193+0000","last_clean":"2024-09-16T11:45:54.566193+0000","last_became_active":"2024-09-16T11:43:31.775335+0000","last_became_peered":"2024-09-16T11:43:31.775335+0000","last_unstale":"2024-09-16T11:45:54.566193+0000","last_undegraded":"2024-09-16T11:45:54.566193+0000","last_fullsized":"2024-09-16T11:45:54.566193+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:07.521713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011352389999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4122,"num_read_kb":15191,"num_write":417,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"132'521","reported_seq":2268,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703740+0000","last_change":"2024-09-16T11:45:47.465501+0000","last_active":"2024-09-16T11:45:53.703740+0000","last_peered":"2024-09-16T11:45:53.703740+0000","last_clean":"2024-09-16T11:45:53.703740+0000","last_became_active":"2024-09-16T11:45:47.463487+0000","last_became_peered":"2024-09-16T11:45:47.463487+0000","last_unstale":"2024-09-16T11:45:53.703740+0000","last_undegraded":"2024-09-16T11:45:53.703740+0000","last_fullsized":"2024-09-16T11:45:53.703740+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:41.597993+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001754511,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4037,"num_read_kb":5871,"num_write":454,"num_write_kb":2926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"125'616","reported_seq":5073,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703823+0000","last_change":"2024-09-16T11:45:47.462731+0000","last_active":"2024-09-16T11:45:53.703823+0000","last_peered":"2024-09-16T11:45:53.703823+0000","last_clean":"2024-09-16T11:45:53.703823+0000","last_became_active":"2024-09-16T11:45:47.461210+0000","last_became_peered":"2024-09-16T11:45:47.461210+0000","last_unstale":"2024-09-16T11:45:53.703823+0000","last_undegraded":"2024-09-16T11:45:53.703823+0000","last_fullsized":"2024-09-16T11:45:53.703823+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:17.838544+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00043040400000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6463,"num_read_kb":36386,"num_write":811,"num_write_kb":14056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"126'605","reported_seq":1931,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703756+0000","last_change":"2024-09-16T11:45:47.465926+0000","last_active":"2024-09-16T11:45:53.703756+0000","last_peered":"2024-09-16T11:45:53.703756+0000","last_clean":"2024-09-16T11:45:53.703756+0000","last_became_active":"2024-09-16T11:45:47.464052+0000","last_became_peered":"2024-09-16T11:45:47.464052+0000","last_unstale":"2024-09-16T11:45:53.703756+0000","last_undegraded":"2024-09-16T11:45:53.703756+0000","last_fullsized":"2024-09-16T11:45:53.703756+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:40.083173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059308799999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2764,"num_read_kb":9887,"num_write":657,"num_write_kb":5333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"127'408","reported_seq":2117,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703980+0000","last_change":"2024-09-16T11:45:39.667068+0000","last_active":"2024-09-16T11:45:53.703980+0000","last_peered":"2024-09-16T11:45:53.703980+0000","last_clean":"2024-09-16T11:45:53.703980+0000","last_became_active":"2024-09-16T11:45:39.643207+0000","last_became_peered":"2024-09-16T11:45:39.643207+0000","last_unstale":"2024-09-16T11:45:53.703980+0000","last_undegraded":"2024-09-16T11:45:53.703980+0000","last_fullsized":"2024-09-16T11:45:53.703980+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:12.700226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070987940000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3609,"num_read_kb":5161,"num_write":362,"num_write_kb":8941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":728,"num_read":135475,"num_read_kb":642896,"num_write":18060,"num_write_kb":290606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},"log_size":16487,"ondisk_log_size":16487,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299128,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072712,"statfs":{"total":96636764160,"available":96330457088,"internally_reserved":0,"allocated":1814528,"data_stored":1481880,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.441,"15min":0.443},"min":{"1min":0.19600000000000001,"5min":0.182,"15min":0.182},"max":{"1min":0.61299999999999999,"5min":0.628,"15min":0.628},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.66000000000000003}]},{"osd":1,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.247,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":4.3559999999999999,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.435,"15min":0.435},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.67900000000000005}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.68400000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.623},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":1.532,"15min":1.532},"last":0.60799999999999998}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.5640000000000001,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.26000000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.67500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45400000000000001,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.437,"15min":0.44},"min":{"1min":0.27600000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65200000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.441},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.318,"5min":0.251,"15min":0.251},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.54700000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.377,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.5},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66200000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52900000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315728,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94056112,"statfs":{"total":96636764160,"available":96313458688,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.438,"15min":0.44400000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52800000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58999999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.503}]},{"osd":1,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.33800000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.53100000000000003,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.439,"15min":0.442},"min":{"1min":0.27400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.61799999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.30399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.35899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.63,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.621}]},{"osd":3,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.55800000000000005}]},{"osd":4,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.64700000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64300000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.48699999999999999,"15min":0.498},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.502,"15min":0.51100000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59899999999999998}]}]},{"osd":3,"up_from":131,"seq":562640715782,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302168,"kb_used_data":1728,"kb_used_omap":43,"kb_used_meta":300372,"kb_avail":94069672,"statfs":{"total":96636764160,"available":96327344128,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44228,"internal_metadata":307581756},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315724,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94056116,"statfs":{"total":96636764160,"available":96313462784,"internally_reserved":0,"allocated":1765376,"data_stored":892439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":1.087,"5min":0.495,"15min":0.39700000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":11.218,"5min":11.218,"15min":11.218},"last":0.31}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32200000000000001},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.372,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.36099999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.38100000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72399999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53500000000000003,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66600000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68600000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53900000000000003}]},{"osd":7,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.871,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311540,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94060300,"statfs":{"total":96636764160,"available":96317747200,"internally_reserved":0,"allocated":1744896,"data_stored":885224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.22700000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.59899999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.379},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.42999999999999999,"15min":0.438},"min":{"1min":0.22900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.61099999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.53700000000000003}]},{"osd":1,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.45300000000000001,"15min":0.45700000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.75800000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47599999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48399999999999999,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":1.026,"15min":1.026},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.47499999999999998,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79600000000000004,"5min":1.01,"15min":1.01},"last":0.56100000000000005}]},{"osd":3,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.622}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39800000000000002,"15min":0.40300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54700000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.502},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.27300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.69899999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.45300000000000001,"15min":0.46500000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.755,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.42099999999999999,"15min":0.433},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52400000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.48199999999999998,"15min":0.495},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67100000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303244,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068596,"statfs":{"total":96636764160,"available":96326242304,"internally_reserved":0,"allocated":1769472,"data_stored":892522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.47599999999999998,"15min":0.441},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":5.8399999999999999,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.44900000000000001,"15min":0.45100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.58699999999999997}]},{"osd":1,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.60699999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.48599999999999999,"15min":0.496},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.621,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.435,"5min":0.49099999999999999,"15min":0.5},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.73499999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.70099999999999996}]},{"osd":5,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.42099999999999999,"15min":0.435},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.44700000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.45500000000000002,"15min":0.46899999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.76400000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.505,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.53500000000000003,"15min":0.55100000000000005},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60299999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.81699999999999995}]},{"osd":7,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56599999999999995,"5min":0.873,"15min":0.873},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58999999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299076,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072764,"statfs":{"total":96636764160,"available":96330510336,"internally_reserved":0,"allocated":1761280,"data_stored":892420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.26200000000000001,"5min":0.29399999999999998,"15min":0.29999999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.46500000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.32800000000000001,"15min":0.32500000000000001},"min":{"1min":0.215,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.255}]},{"osd":1,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32900000000000001,"15min":0.33200000000000002},"min":{"1min":0.23000000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.437,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34799999999999998,"15min":0.34899999999999998},"min":{"1min":0.255,"5min":0.192,"15min":0.192},"max":{"1min":0.53600000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.40400000000000003}]},{"osd":4,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.52700000000000002,"15min":0.54100000000000004},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.35599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.216}]},{"osd":5,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":1.254}]},{"osd":6,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57999999999999996,"15min":0.58899999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":1.288},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.57799999999999996,"15min":0.58799999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.3089999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.266}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299140,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072700,"statfs":{"total":96636764160,"available":96330444800,"internally_reserved":0,"allocated":1822720,"data_stored":1482871,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.39600000000000002,"15min":0.33800000000000002},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":8.5050000000000008,"5min":8.5050000000000008,"15min":8.5050000000000008},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32100000000000001},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.436,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.26200000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.39600000000000002,"15min":0.33600000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":8.4689999999999994,"5min":8.4689999999999994,"15min":8.4689999999999994},"last":0.315},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.23699999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47699999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.30399999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.27700000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.57399999999999995,"15min":0.51400000000000001},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":8.5299999999999994,"5min":8.5299999999999994,"15min":8.5299999999999994},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.503,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71199999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.496}]},{"osd":5,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.65100000000000002,"15min":0.60299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":8.6039999999999992,"5min":8.6039999999999992,"15min":8.6039999999999992},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.55300000000000005,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69099999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.39500000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.64200000000000002,"5min":1.075,"15min":1.075},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.67500000000000004,"15min":0.628},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":8.5809999999999995,"5min":8.5809999999999995,"15min":8.5809999999999995},"last":0.46800000000000003}]},{"osd":7,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64800000000000002,"5min":0.747,"15min":0.747},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.61199999999999999,"15min":0.55200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":8.5570000000000004,"5min":8.5570000000000004,"15min":8.5570000000000004},"last":0.42399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:46:00.867 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T11:46:00.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:46:01.129 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:46:01.130 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:46:01.141 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-16T11:45:59.402327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590574,"num_objects":741,"num_object_clones":0,"num_object_copies":1482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":731,"num_read":135541,"num_read_kb":642951,"num_write":18165,"num_write_kb":292238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16567,"ondisk_log_size":16567,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445748,"kb_used_data":13884,"kb_used_omap":54,"kb_used_meta":2431689,"kb_avail":752528972,"statfs":{"total":773094113280,"available":770589667328,"internally_reserved":0,"allocated":14217216,"data_stored":8312404,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56156,"internal_metadata":2490049700},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.705432"},"pg_stats":[{"pgid":"4.1f","version":"127'327","reported_seq":6237,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725885+0000","last_change":"2024-09-16T11:45:19.597844+0000","last_active":"2024-09-16T11:45:54.725885+0000","last_peered":"2024-09-16T11:45:54.725885+0000","last_clean":"2024-09-16T11:45:54.725885+0000","last_became_active":"2024-09-16T11:43:31.779428+0000","last_became_peered":"2024-09-16T11:43:31.779428+0000","last_unstale":"2024-09-16T11:45:54.725885+0000","last_undegraded":"2024-09-16T11:45:54.725885+0000","last_fullsized":"2024-09-16T11:45:54.725885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:30.437698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034041099999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6762,"num_read_kb":29123,"num_write":304,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"125'449","reported_seq":5346,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566266+0000","last_change":"2024-09-16T11:45:19.592718+0000","last_active":"2024-09-16T11:45:54.566266+0000","last_peered":"2024-09-16T11:45:54.566266+0000","last_clean":"2024-09-16T11:45:54.566266+0000","last_became_active":"2024-09-16T11:43:31.889706+0000","last_became_peered":"2024-09-16T11:43:31.889706+0000","last_unstale":"2024-09-16T11:45:54.566266+0000","last_undegraded":"2024-09-16T11:45:54.566266+0000","last_fullsized":"2024-09-16T11:45:54.566266+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:25.242116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000654113,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5672,"num_read_kb":36055,"num_write":486,"num_write_kb":9211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"132'213","reported_seq":1539,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725493+0000","last_change":"2024-09-16T11:45:39.661471+0000","last_active":"2024-09-16T11:45:54.725493+0000","last_peered":"2024-09-16T11:45:54.725493+0000","last_clean":"2024-09-16T11:45:54.725493+0000","last_became_active":"2024-09-16T11:45:39.636105+0000","last_became_peered":"2024-09-16T11:45:39.636105+0000","last_unstale":"2024-09-16T11:45:54.725493+0000","last_undegraded":"2024-09-16T11:45:54.725493+0000","last_fullsized":"2024-09-16T11:45:54.725493+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:18:52.260065+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.014051187999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1102,"num_read_kb":9776,"num_write":200,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"127'334","reported_seq":2413,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703281+0000","last_change":"2024-09-16T11:45:19.592554+0000","last_active":"2024-09-16T11:45:53.703281+0000","last_peered":"2024-09-16T11:45:53.703281+0000","last_clean":"2024-09-16T11:45:53.703281+0000","last_became_active":"2024-09-16T11:43:31.773661+0000","last_became_peered":"2024-09-16T11:43:31.773661+0000","last_unstale":"2024-09-16T11:45:53.703281+0000","last_undegraded":"2024-09-16T11:45:53.703281+0000","last_fullsized":"2024-09-16T11:45:53.703281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:36.062796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052992100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":2191,"num_read_kb":14785,"num_write":375,"num_write_kb":7446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"132'380","reported_seq":2524,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703902+0000","last_change":"2024-09-16T11:45:47.464749+0000","last_active":"2024-09-16T11:45:53.703902+0000","last_peered":"2024-09-16T11:45:53.703902+0000","last_clean":"2024-09-16T11:45:53.703902+0000","last_became_active":"2024-09-16T11:45:47.463209+0000","last_became_peered":"2024-09-16T11:45:47.463209+0000","last_unstale":"2024-09-16T11:45:53.703902+0000","last_undegraded":"2024-09-16T11:45:53.703902+0000","last_fullsized":"2024-09-16T11:45:53.703902+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:29.503448+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0033897049999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3246,"num_read_kb":18406,"num_write":318,"num_write_kb":4718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"132'534","reported_seq":3517,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703670+0000","last_change":"2024-09-16T11:45:47.464540+0000","last_active":"2024-09-16T11:45:53.703670+0000","last_peered":"2024-09-16T11:45:53.703670+0000","last_clean":"2024-09-16T11:45:53.703670+0000","last_became_active":"2024-09-16T11:45:47.463199+0000","last_became_peered":"2024-09-16T11:45:47.463199+0000","last_unstale":"2024-09-16T11:45:53.703670+0000","last_undegraded":"2024-09-16T11:45:53.703670+0000","last_fullsized":"2024-09-16T11:45:53.703670+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:44.772848+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.01209723,"stat_sum":{"num_bytes":19,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":4102,"num_read_kb":12532,"num_write":637,"num_write_kb":9975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"124'658","reported_seq":4607,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725383+0000","last_change":"2024-09-16T11:45:39.678403+0000","last_active":"2024-09-16T11:45:54.725383+0000","last_peered":"2024-09-16T11:45:54.725383+0000","last_clean":"2024-09-16T11:45:54.725383+0000","last_became_active":"2024-09-16T11:45:39.638104+0000","last_became_peered":"2024-09-16T11:45:39.638104+0000","last_unstale":"2024-09-16T11:45:54.725383+0000","last_undegraded":"2024-09-16T11:45:54.725383+0000","last_fullsized":"2024-09-16T11:45:54.725383+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:18.828819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069266390000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5632,"num_read_kb":29340,"num_write":868,"num_write_kb":4351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"127'221","reported_seq":1540,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703616+0000","last_change":"2024-09-16T11:45:19.592170+0000","last_active":"2024-09-16T11:45:53.703616+0000","last_peered":"2024-09-16T11:45:53.703616+0000","last_clean":"2024-09-16T11:45:53.703616+0000","last_became_active":"2024-09-16T11:43:31.776951+0000","last_became_peered":"2024-09-16T11:43:31.776951+0000","last_unstale":"2024-09-16T11:45:53.703616+0000","last_undegraded":"2024-09-16T11:45:53.703616+0000","last_fullsized":"2024-09-16T11:45:53.703616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:28:39.026621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052508100000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1522,"num_read_kb":7050,"num_write":185,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"132'525","reported_seq":2341,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725539+0000","last_change":"2024-09-16T11:45:39.723943+0000","last_active":"2024-09-16T11:45:54.725539+0000","last_peered":"2024-09-16T11:45:54.725539+0000","last_clean":"2024-09-16T11:45:54.725539+0000","last_became_active":"2024-09-16T11:45:39.642890+0000","last_became_peered":"2024-09-16T11:45:39.642890+0000","last_unstale":"2024-09-16T11:45:54.725539+0000","last_undegraded":"2024-09-16T11:45:54.725539+0000","last_fullsized":"2024-09-16T11:45:54.725539+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:26.709287+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062520060000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4283,"num_read_kb":14493,"num_write":541,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"132'743","reported_seq":6417,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566264+0000","last_change":"2024-09-16T11:45:39.656713+0000","last_active":"2024-09-16T11:45:54.566264+0000","last_peered":"2024-09-16T11:45:54.566264+0000","last_clean":"2024-09-16T11:45:54.566264+0000","last_became_active":"2024-09-16T11:45:39.640293+0000","last_became_peered":"2024-09-16T11:45:39.640293+0000","last_unstale":"2024-09-16T11:45:54.566264+0000","last_undegraded":"2024-09-16T11:45:54.566264+0000","last_fullsized":"2024-09-16T11:45:54.566264+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:29:01.940843+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074727580000000004,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8298,"num_read_kb":47234,"num_write":833,"num_write_kb":15319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"127'657","reported_seq":2814,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725859+0000","last_change":"2024-09-16T11:45:19.636993+0000","last_active":"2024-09-16T11:45:54.725859+0000","last_peered":"2024-09-16T11:45:54.725859+0000","last_clean":"2024-09-16T11:45:54.725859+0000","last_became_active":"2024-09-16T11:43:31.889629+0000","last_became_peered":"2024-09-16T11:43:31.889629+0000","last_unstale":"2024-09-16T11:45:54.725859+0000","last_undegraded":"2024-09-16T11:45:54.725859+0000","last_fullsized":"2024-09-16T11:45:54.725859+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:53.124562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030057499999999997,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":3982,"num_read_kb":26490,"num_write":811,"num_write_kb":16305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"132'665","reported_seq":3461,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725390+0000","last_change":"2024-09-16T11:45:39.707640+0000","last_active":"2024-09-16T11:45:54.725390+0000","last_peered":"2024-09-16T11:45:54.725390+0000","last_clean":"2024-09-16T11:45:54.725390+0000","last_became_active":"2024-09-16T11:45:39.641187+0000","last_became_peered":"2024-09-16T11:45:39.641187+0000","last_unstale":"2024-09-16T11:45:54.725390+0000","last_undegraded":"2024-09-16T11:45:54.725390+0000","last_fullsized":"2024-09-16T11:45:54.725390+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:47:39.464422+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074563379999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4363,"num_read_kb":20582,"num_write":914,"num_write_kb":10744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":3197941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"127'546","reported_seq":2511,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703790+0000","last_change":"2024-09-16T11:45:19.592153+0000","last_active":"2024-09-16T11:45:53.703790+0000","last_peered":"2024-09-16T11:45:53.703790+0000","last_clean":"2024-09-16T11:45:53.703790+0000","last_became_active":"2024-09-16T11:43:31.776781+0000","last_became_peered":"2024-09-16T11:43:31.776781+0000","last_unstale":"2024-09-16T11:45:53.703790+0000","last_undegraded":"2024-09-16T11:45:53.703790+0000","last_fullsized":"2024-09-16T11:45:53.703790+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:14.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032933499999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3911,"num_read_kb":15179,"num_write":595,"num_write_kb":10620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"121'294","reported_seq":1908,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703771+0000","last_change":"2024-09-16T11:45:19.642323+0000","last_active":"2024-09-16T11:45:53.703771+0000","last_peered":"2024-09-16T11:45:53.703771+0000","last_clean":"2024-09-16T11:45:53.703771+0000","last_became_active":"2024-09-16T11:43:31.780164+0000","last_became_peered":"2024-09-16T11:43:31.780164+0000","last_unstale":"2024-09-16T11:45:53.703771+0000","last_undegraded":"2024-09-16T11:45:53.703771+0000","last_fullsized":"2024-09-16T11:45:53.703771+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:56.846891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079981,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1687,"num_read_kb":10324,"num_write":299,"num_write_kb":6058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"132'804","reported_seq":2260,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725294+0000","last_change":"2024-09-16T11:45:39.712869+0000","last_active":"2024-09-16T11:45:54.725294+0000","last_peered":"2024-09-16T11:45:54.725294+0000","last_clean":"2024-09-16T11:45:54.725294+0000","last_became_active":"2024-09-16T11:45:39.641126+0000","last_became_peered":"2024-09-16T11:45:39.641126+0000","last_unstale":"2024-09-16T11:45:54.725294+0000","last_undegraded":"2024-09-16T11:45:54.725294+0000","last_fullsized":"2024-09-16T11:45:54.725294+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:23.732386+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074597470000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":4259,"num_read_kb":18254,"num_write":1055,"num_write_kb":13061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"132'401","reported_seq":1908,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725271+0000","last_change":"2024-09-16T11:45:39.666850+0000","last_active":"2024-09-16T11:45:54.725271+0000","last_peered":"2024-09-16T11:45:54.725271+0000","last_clean":"2024-09-16T11:45:54.725271+0000","last_became_active":"2024-09-16T11:45:39.637293+0000","last_became_peered":"2024-09-16T11:45:39.637293+0000","last_unstale":"2024-09-16T11:45:54.725271+0000","last_undegraded":"2024-09-16T11:45:54.725271+0000","last_fullsized":"2024-09-16T11:45:54.725271+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:39.210719+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.016205368000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3249,"num_read_kb":9637,"num_write":346,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725801+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:45:54.725801+0000","last_peered":"2024-09-16T11:45:54.725801+0000","last_clean":"2024-09-16T11:45:54.725801+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:45:54.725801+0000","last_undegraded":"2024-09-16T11:45:54.725801+0000","last_fullsized":"2024-09-16T11:45:54.725801+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"125'347","reported_seq":1761,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703661+0000","last_change":"2024-09-16T11:45:47.465149+0000","last_active":"2024-09-16T11:45:53.703661+0000","last_peered":"2024-09-16T11:45:53.703661+0000","last_clean":"2024-09-16T11:45:53.703661+0000","last_became_active":"2024-09-16T11:45:47.464115+0000","last_became_peered":"2024-09-16T11:45:47.464115+0000","last_unstale":"2024-09-16T11:45:53.703661+0000","last_undegraded":"2024-09-16T11:45:53.703661+0000","last_fullsized":"2024-09-16T11:45:53.703661+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:34.356992+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00038392500000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2491,"num_read_kb":9150,"num_write":283,"num_write_kb":9998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703962+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:45:53.703962+0000","last_peered":"2024-09-16T11:45:53.703962+0000","last_clean":"2024-09-16T11:45:53.703962+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:45:53.703962+0000","last_undegraded":"2024-09-16T11:45:53.703962+0000","last_fullsized":"2024-09-16T11:45:53.703962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"132'477","reported_seq":4581,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725271+0000","last_change":"2024-09-16T11:45:39.723814+0000","last_active":"2024-09-16T11:45:54.725271+0000","last_peered":"2024-09-16T11:45:54.725271+0000","last_clean":"2024-09-16T11:45:54.725271+0000","last_became_active":"2024-09-16T11:45:39.641298+0000","last_became_peered":"2024-09-16T11:45:39.641298+0000","last_unstale":"2024-09-16T11:45:54.725271+0000","last_undegraded":"2024-09-16T11:45:54.725271+0000","last_fullsized":"2024-09-16T11:45:54.725271+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:57.805852+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0094940819999999992,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5544,"num_read_kb":23694,"num_write":515,"num_write_kb":8223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.722985+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:45:53.722985+0000","last_peered":"2024-09-16T11:45:53.722985+0000","last_clean":"2024-09-16T11:45:53.722985+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:45:53.722985+0000","last_undegraded":"2024-09-16T11:45:53.722985+0000","last_fullsized":"2024-09-16T11:45:53.722985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"127'627","reported_seq":6867,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566281+0000","last_change":"2024-09-16T11:45:19.593742+0000","last_active":"2024-09-16T11:45:54.566281+0000","last_peered":"2024-09-16T11:45:54.566281+0000","last_clean":"2024-09-16T11:45:54.566281+0000","last_became_active":"2024-09-16T11:43:31.777896+0000","last_became_peered":"2024-09-16T11:43:31.777896+0000","last_unstale":"2024-09-16T11:45:54.566281+0000","last_undegraded":"2024-09-16T11:45:54.566281+0000","last_fullsized":"2024-09-16T11:45:54.566281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:35.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011770719999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":7410,"num_read_kb":47228,"num_write":617,"num_write_kb":8894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704146+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:45:53.704146+0000","last_peered":"2024-09-16T11:45:53.704146+0000","last_clean":"2024-09-16T11:45:53.704146+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:45:53.704146+0000","last_undegraded":"2024-09-16T11:45:53.704146+0000","last_fullsized":"2024-09-16T11:45:53.704146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"126'382","reported_seq":1083,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703876+0000","last_change":"2024-09-16T11:45:19.642384+0000","last_active":"2024-09-16T11:45:53.703876+0000","last_peered":"2024-09-16T11:45:53.703876+0000","last_clean":"2024-09-16T11:45:53.703876+0000","last_became_active":"2024-09-16T11:43:31.778506+0000","last_became_peered":"2024-09-16T11:43:31.778506+0000","last_unstale":"2024-09-16T11:45:53.703876+0000","last_undegraded":"2024-09-16T11:45:53.703876+0000","last_fullsized":"2024-09-16T11:45:53.703876+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:12.232493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011478079999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1438,"num_read_kb":4478,"num_write":491,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":110,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703900+0000","last_change":"2024-09-16T11:45:47.462953+0000","last_active":"2024-09-16T11:45:53.703900+0000","last_peered":"2024-09-16T11:45:53.703900+0000","last_clean":"2024-09-16T11:45:53.703900+0000","last_became_active":"2024-09-16T11:45:47.462295+0000","last_became_peered":"2024-09-16T11:45:47.462295+0000","last_unstale":"2024-09-16T11:45:53.703900+0000","last_undegraded":"2024-09-16T11:45:53.703900+0000","last_fullsized":"2024-09-16T11:45:53.703900+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:57.786498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":140,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725763+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:45:54.725763+0000","last_peered":"2024-09-16T11:45:54.725763+0000","last_clean":"2024-09-16T11:45:54.725763+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:45:54.725763+0000","last_undegraded":"2024-09-16T11:45:54.725763+0000","last_fullsized":"2024-09-16T11:45:54.725763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"127'734","reported_seq":2603,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704134+0000","last_change":"2024-09-16T11:45:19.641341+0000","last_active":"2024-09-16T11:45:53.704134+0000","last_peered":"2024-09-16T11:45:53.704134+0000","last_clean":"2024-09-16T11:45:53.704134+0000","last_became_active":"2024-09-16T11:43:31.777796+0000","last_became_peered":"2024-09-16T11:43:31.777796+0000","last_unstale":"2024-09-16T11:45:53.704134+0000","last_undegraded":"2024-09-16T11:45:53.704134+0000","last_fullsized":"2024-09-16T11:45:53.704134+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:33.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051098400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4214,"num_read_kb":15199,"num_write":913,"num_write_kb":10139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":132,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704039+0000","last_change":"2024-09-16T11:43:22.440136+0000","last_active":"2024-09-16T11:45:53.704039+0000","last_peered":"2024-09-16T11:45:53.704039+0000","last_clean":"2024-09-16T11:45:53.704039+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:45:53.704039+0000","last_undegraded":"2024-09-16T11:45:53.704039+0000","last_fullsized":"2024-09-16T11:45:53.704039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:18.461948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703187+0000","last_change":"2024-09-16T11:45:39.635580+0000","last_active":"2024-09-16T11:45:53.703187+0000","last_peered":"2024-09-16T11:45:53.703187+0000","last_clean":"2024-09-16T11:45:53.703187+0000","last_became_active":"2024-09-16T11:45:39.635334+0000","last_became_peered":"2024-09-16T11:45:39.635334+0000","last_unstale":"2024-09-16T11:45:53.703187+0000","last_undegraded":"2024-09-16T11:45:53.703187+0000","last_fullsized":"2024-09-16T11:45:53.703187+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:43.033024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"125'546","reported_seq":3390,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704084+0000","last_change":"2024-09-16T11:45:19.641909+0000","last_active":"2024-09-16T11:45:53.704084+0000","last_peered":"2024-09-16T11:45:53.704084+0000","last_clean":"2024-09-16T11:45:53.704084+0000","last_became_active":"2024-09-16T11:43:31.777967+0000","last_became_peered":"2024-09-16T11:43:31.777967+0000","last_unstale":"2024-09-16T11:45:53.704084+0000","last_undegraded":"2024-09-16T11:45:53.704084+0000","last_fullsized":"2024-09-16T11:45:53.704084+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:43.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081483900000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3645,"num_read_kb":23388,"num_write":630,"num_write_kb":9618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"112'2","reported_seq":133,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566166+0000","last_change":"2024-09-16T11:43:22.437965+0000","last_active":"2024-09-16T11:45:54.566166+0000","last_peered":"2024-09-16T11:45:54.566166+0000","last_clean":"2024-09-16T11:45:54.566166+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:45:54.566166+0000","last_undegraded":"2024-09-16T11:45:54.566166+0000","last_fullsized":"2024-09-16T11:45:54.566166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:52.884360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.723066+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:45:53.723066+0000","last_peered":"2024-09-16T11:45:53.723066+0000","last_clean":"2024-09-16T11:45:53.723066+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:45:53.723066+0000","last_undegraded":"2024-09-16T11:45:53.723066+0000","last_fullsized":"2024-09-16T11:45:53.723066+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"127'407","reported_seq":1770,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566334+0000","last_change":"2024-09-16T11:45:19.592986+0000","last_active":"2024-09-16T11:45:54.566334+0000","last_peered":"2024-09-16T11:45:54.566334+0000","last_clean":"2024-09-16T11:45:54.566334+0000","last_became_active":"2024-09-16T11:43:31.776645+0000","last_became_peered":"2024-09-16T11:43:31.776645+0000","last_unstale":"2024-09-16T11:45:54.566334+0000","last_undegraded":"2024-09-16T11:45:54.566334+0000","last_fullsized":"2024-09-16T11:45:54.566334+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:53.198323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066659399999999995,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1650,"num_read_kb":12114,"num_write":331,"num_write_kb":15245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"84'3","reported_seq":97,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725388+0000","last_change":"2024-09-16T11:45:39.638395+0000","last_active":"2024-09-16T11:45:54.725388+0000","last_peered":"2024-09-16T11:45:54.725388+0000","last_clean":"2024-09-16T11:45:54.725388+0000","last_became_active":"2024-09-16T11:45:39.637629+0000","last_became_peered":"2024-09-16T11:45:39.637629+0000","last_unstale":"2024-09-16T11:45:54.725388+0000","last_undegraded":"2024-09-16T11:45:54.725388+0000","last_fullsized":"2024-09-16T11:45:54.725388+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:56.174857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725989+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:45:54.725989+0000","last_peered":"2024-09-16T11:45:54.725989+0000","last_clean":"2024-09-16T11:45:54.725989+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:45:54.725989+0000","last_undegraded":"2024-09-16T11:45:54.725989+0000","last_fullsized":"2024-09-16T11:45:54.725989+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":226,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.723073+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:45:53.723073+0000","last_peered":"2024-09-16T11:45:53.723073+0000","last_clean":"2024-09-16T11:45:53.723073+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:45:53.723073+0000","last_undegraded":"2024-09-16T11:45:53.723073+0000","last_fullsized":"2024-09-16T11:45:53.723073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"125'523","reported_seq":3438,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725834+0000","last_change":"2024-09-16T11:45:19.598172+0000","last_active":"2024-09-16T11:45:54.725834+0000","last_peered":"2024-09-16T11:45:54.725834+0000","last_clean":"2024-09-16T11:45:54.725834+0000","last_became_active":"2024-09-16T11:43:31.782071+0000","last_became_peered":"2024-09-16T11:43:31.782071+0000","last_unstale":"2024-09-16T11:45:54.725834+0000","last_undegraded":"2024-09-16T11:45:54.725834+0000","last_fullsized":"2024-09-16T11:45:54.725834+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:15.516215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000599604,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3801,"num_read_kb":24361,"num_write":540,"num_write_kb":7978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"127'665","reported_seq":9006,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703769+0000","last_change":"2024-09-16T11:45:19.641414+0000","last_active":"2024-09-16T11:45:53.703769+0000","last_peered":"2024-09-16T11:45:53.703769+0000","last_clean":"2024-09-16T11:45:53.703769+0000","last_became_active":"2024-09-16T11:44:10.920131+0000","last_became_peered":"2024-09-16T11:44:10.920131+0000","last_unstale":"2024-09-16T11:45:53.703769+0000","last_undegraded":"2024-09-16T11:45:53.703769+0000","last_fullsized":"2024-09-16T11:45:53.703769+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:55:10.158414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082161099999999998,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":9498,"num_read_kb":69976,"num_write":712,"num_write_kb":19149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":8960357,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"127'878","reported_seq":3861,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703624+0000","last_change":"2024-09-16T11:45:47.465826+0000","last_active":"2024-09-16T11:45:53.703624+0000","last_peered":"2024-09-16T11:45:53.703624+0000","last_clean":"2024-09-16T11:45:53.703624+0000","last_became_active":"2024-09-16T11:45:47.464353+0000","last_became_peered":"2024-09-16T11:45:47.464353+0000","last_unstale":"2024-09-16T11:45:53.703624+0000","last_undegraded":"2024-09-16T11:45:53.703624+0000","last_fullsized":"2024-09-16T11:45:53.703624+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:24.438451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001068741,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":8005,"num_read_kb":12353,"num_write":894,"num_write_kb":6127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"127'623","reported_seq":1817,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.726002+0000","last_change":"2024-09-16T11:45:19.598161+0000","last_active":"2024-09-16T11:45:54.726002+0000","last_peered":"2024-09-16T11:45:54.726002+0000","last_clean":"2024-09-16T11:45:54.726002+0000","last_became_active":"2024-09-16T11:43:31.779384+0000","last_became_peered":"2024-09-16T11:43:31.779384+0000","last_unstale":"2024-09-16T11:45:54.726002+0000","last_undegraded":"2024-09-16T11:45:54.726002+0000","last_fullsized":"2024-09-16T11:45:54.726002+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:34.933146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000647592,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":2523,"num_read_kb":9199,"num_write":666,"num_write_kb":6203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"125'377","reported_seq":3380,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566193+0000","last_change":"2024-09-16T11:45:19.593453+0000","last_active":"2024-09-16T11:45:54.566193+0000","last_peered":"2024-09-16T11:45:54.566193+0000","last_clean":"2024-09-16T11:45:54.566193+0000","last_became_active":"2024-09-16T11:43:31.775335+0000","last_became_peered":"2024-09-16T11:43:31.775335+0000","last_unstale":"2024-09-16T11:45:54.566193+0000","last_undegraded":"2024-09-16T11:45:54.566193+0000","last_fullsized":"2024-09-16T11:45:54.566193+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:07.521713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011352389999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4122,"num_read_kb":15191,"num_write":417,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"132'521","reported_seq":2268,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703740+0000","last_change":"2024-09-16T11:45:47.465501+0000","last_active":"2024-09-16T11:45:53.703740+0000","last_peered":"2024-09-16T11:45:53.703740+0000","last_clean":"2024-09-16T11:45:53.703740+0000","last_became_active":"2024-09-16T11:45:47.463487+0000","last_became_peered":"2024-09-16T11:45:47.463487+0000","last_unstale":"2024-09-16T11:45:53.703740+0000","last_undegraded":"2024-09-16T11:45:53.703740+0000","last_fullsized":"2024-09-16T11:45:53.703740+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:41.597993+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001754511,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4037,"num_read_kb":5871,"num_write":454,"num_write_kb":2926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"125'616","reported_seq":5073,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703823+0000","last_change":"2024-09-16T11:45:47.462731+0000","last_active":"2024-09-16T11:45:53.703823+0000","last_peered":"2024-09-16T11:45:53.703823+0000","last_clean":"2024-09-16T11:45:53.703823+0000","last_became_active":"2024-09-16T11:45:47.461210+0000","last_became_peered":"2024-09-16T11:45:47.461210+0000","last_unstale":"2024-09-16T11:45:53.703823+0000","last_undegraded":"2024-09-16T11:45:53.703823+0000","last_fullsized":"2024-09-16T11:45:53.703823+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:17.838544+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00043040400000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6463,"num_read_kb":36386,"num_write":811,"num_write_kb":14056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"126'605","reported_seq":1931,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703756+0000","last_change":"2024-09-16T11:45:47.465926+0000","last_active":"2024-09-16T11:45:53.703756+0000","last_peered":"2024-09-16T11:45:53.703756+0000","last_clean":"2024-09-16T11:45:53.703756+0000","last_became_active":"2024-09-16T11:45:47.464052+0000","last_became_peered":"2024-09-16T11:45:47.464052+0000","last_unstale":"2024-09-16T11:45:53.703756+0000","last_undegraded":"2024-09-16T11:45:53.703756+0000","last_fullsized":"2024-09-16T11:45:53.703756+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:40.083173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059308799999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2764,"num_read_kb":9887,"num_write":657,"num_write_kb":5333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"127'408","reported_seq":2117,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703980+0000","last_change":"2024-09-16T11:45:39.667068+0000","last_active":"2024-09-16T11:45:53.703980+0000","last_peered":"2024-09-16T11:45:53.703980+0000","last_clean":"2024-09-16T11:45:53.703980+0000","last_became_active":"2024-09-16T11:45:39.643207+0000","last_became_peered":"2024-09-16T11:45:39.643207+0000","last_unstale":"2024-09-16T11:45:53.703980+0000","last_undegraded":"2024-09-16T11:45:53.703980+0000","last_fullsized":"2024-09-16T11:45:53.703980+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:12.700226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070987940000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3609,"num_read_kb":5161,"num_write":362,"num_write_kb":8941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":728,"num_read":135475,"num_read_kb":642896,"num_write":18060,"num_write_kb":290606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},"log_size":16487,"ondisk_log_size":16487,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299128,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072712,"statfs":{"total":96636764160,"available":96330457088,"internally_reserved":0,"allocated":1814528,"data_stored":1481880,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.441,"15min":0.443},"min":{"1min":0.19600000000000001,"5min":0.182,"15min":0.182},"max":{"1min":0.61299999999999999,"5min":0.628,"15min":0.628},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.66000000000000003}]},{"osd":1,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.247,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":4.3559999999999999,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.435,"15min":0.435},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.67900000000000005}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.68400000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.623},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":1.532,"15min":1.532},"last":0.60799999999999998}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.5640000000000001,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.26000000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.67500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45400000000000001,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.437,"15min":0.44},"min":{"1min":0.27600000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65200000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.441},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.318,"5min":0.251,"15min":0.251},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.54700000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.377,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.5},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66200000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52900000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315728,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94056112,"statfs":{"total":96636764160,"available":96313458688,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.438,"15min":0.44400000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52800000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58999999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.503}]},{"osd":1,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.33800000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.53100000000000003,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.439,"15min":0.442},"min":{"1min":0.27400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.61799999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.30399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.35899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.63,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.621}]},{"osd":3,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.55800000000000005}]},{"osd":4,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.64700000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64300000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.48699999999999999,"15min":0.498},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.502,"15min":0.51100000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59899999999999998}]}]},{"osd":3,"up_from":131,"seq":562640715782,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302168,"kb_used_data":1728,"kb_used_omap":43,"kb_used_meta":300372,"kb_avail":94069672,"statfs":{"total":96636764160,"available":96327344128,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44228,"internal_metadata":307581756},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315724,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94056116,"statfs":{"total":96636764160,"available":96313462784,"internally_reserved":0,"allocated":1765376,"data_stored":892439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":1.087,"5min":0.495,"15min":0.39700000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":11.218,"5min":11.218,"15min":11.218},"last":0.31}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32200000000000001},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.372,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.36099999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.38100000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72399999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53500000000000003,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66600000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68600000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53900000000000003}]},{"osd":7,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.871,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311540,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94060300,"statfs":{"total":96636764160,"available":96317747200,"internally_reserved":0,"allocated":1744896,"data_stored":885224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.22700000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.59899999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.379},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.42999999999999999,"15min":0.438},"min":{"1min":0.22900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.61099999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.53700000000000003}]},{"osd":1,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.45300000000000001,"15min":0.45700000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.75800000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47599999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48399999999999999,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":1.026,"15min":1.026},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.47499999999999998,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79600000000000004,"5min":1.01,"15min":1.01},"last":0.56100000000000005}]},{"osd":3,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.622}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39800000000000002,"15min":0.40300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54700000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.502},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.27300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.69899999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.45300000000000001,"15min":0.46500000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.755,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.42099999999999999,"15min":0.433},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52400000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.48199999999999998,"15min":0.495},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67100000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303244,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068596,"statfs":{"total":96636764160,"available":96326242304,"internally_reserved":0,"allocated":1769472,"data_stored":892522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.47599999999999998,"15min":0.441},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":5.8399999999999999,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.44900000000000001,"15min":0.45100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.58699999999999997}]},{"osd":1,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.60699999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.48599999999999999,"15min":0.496},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.621,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.435,"5min":0.49099999999999999,"15min":0.5},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.73499999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.70099999999999996}]},{"osd":5,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.42099999999999999,"15min":0.435},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.44700000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.45500000000000002,"15min":0.46899999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.76400000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.505,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.53500000000000003,"15min":0.55100000000000005},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60299999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.81699999999999995}]},{"osd":7,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56599999999999995,"5min":0.873,"15min":0.873},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58999999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299076,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072764,"statfs":{"total":96636764160,"available":96330510336,"internally_reserved":0,"allocated":1761280,"data_stored":892420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.26200000000000001,"5min":0.29399999999999998,"15min":0.29999999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.46500000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.32800000000000001,"15min":0.32500000000000001},"min":{"1min":0.215,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.255}]},{"osd":1,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32900000000000001,"15min":0.33200000000000002},"min":{"1min":0.23000000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.437,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34799999999999998,"15min":0.34899999999999998},"min":{"1min":0.255,"5min":0.192,"15min":0.192},"max":{"1min":0.53600000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.40400000000000003}]},{"osd":4,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.52700000000000002,"15min":0.54100000000000004},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.35599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.216}]},{"osd":5,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":1.254}]},{"osd":6,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57999999999999996,"15min":0.58899999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":1.288},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.57799999999999996,"15min":0.58799999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.3089999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.266}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299140,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072700,"statfs":{"total":96636764160,"available":96330444800,"internally_reserved":0,"allocated":1822720,"data_stored":1482871,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.39600000000000002,"15min":0.33800000000000002},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":8.5050000000000008,"5min":8.5050000000000008,"15min":8.5050000000000008},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32100000000000001},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.436,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.26200000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.39600000000000002,"15min":0.33600000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":8.4689999999999994,"5min":8.4689999999999994,"15min":8.4689999999999994},"last":0.315},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.23699999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47699999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.30399999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.27700000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.57399999999999995,"15min":0.51400000000000001},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":8.5299999999999994,"5min":8.5299999999999994,"15min":8.5299999999999994},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.503,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71199999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.496}]},{"osd":5,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.65100000000000002,"15min":0.60299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":8.6039999999999992,"5min":8.6039999999999992,"15min":8.6039999999999992},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.55300000000000005,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69099999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.39500000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.64200000000000002,"5min":1.075,"15min":1.075},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.67500000000000004,"15min":0.628},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":8.5809999999999995,"5min":8.5809999999999995,"15min":8.5809999999999995},"last":0.46800000000000003}]},{"osd":7,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64800000000000002,"5min":0.747,"15min":0.747},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.61199999999999999,"15min":0.55200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":8.5570000000000004,"5min":8.5570000000000004,"15min":8.5570000000000004},"last":0.42399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:46:01.143 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:46:01.403 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:46:01.403 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":147,"fsid":"3420bcad-625f-4e15-aaff-fb4236322168","created":"2024-09-16T11:43:01.654626+0000","modified":"2024-09-16T11:45:54.705133+0000","last_up_change":"2024-09-16T11:45:38.626538+0000","last_in_change":"2024-09-16T11:43:04.526236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:43:11.406582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:43:13.125709+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"23","last_force_op_resend_prenautilus":"23","last_force_op_resend_preluminous":"23","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T11:43:21.227888+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"23","last_force_op_resend_prenautilus":"23","last_force_op_resend_preluminous":"23","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T11:43:30.105533+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":83,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"91f052e4-4c8f-4484-92ce-bd1a7f8f3efb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":33355},{"type":"v1","addr":"172.21.15.70:6801","nonce":33355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":33355},{"type":"v1","addr":"172.21.15.70:6803","nonce":33355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":33355},{"type":"v1","addr":"172.21.15.70:6807","nonce":33355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":33355},{"type":"v1","addr":"172.21.15.70:6805","nonce":33355}]},"public_addr":"172.21.15.70:6801/33355","cluster_addr":"172.21.15.70:6803/33355","heartbeat_back_addr":"172.21.15.70:6807/33355","heartbeat_front_addr":"172.21.15.70:6805/33355","state":["exists","up"]},{"osd":1,"uuid":"568b1280-7365-4094-afe8-b0b5e766de6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":33359},{"type":"v1","addr":"172.21.15.70:6817","nonce":33359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":33359},{"type":"v1","addr":"172.21.15.70:6819","nonce":33359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":33359},{"type":"v1","addr":"172.21.15.70:6823","nonce":33359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":33359},{"type":"v1","addr":"172.21.15.70:6821","nonce":33359}]},"public_addr":"172.21.15.70:6817/33359","cluster_addr":"172.21.15.70:6819/33359","heartbeat_back_addr":"172.21.15.70:6823/33359","heartbeat_front_addr":"172.21.15.70:6821/33359","state":["exists","up"]},{"osd":2,"uuid":"328adfa3-c1b4-4745-9759-7327c8cbd499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":33356},{"type":"v1","addr":"172.21.15.70:6809","nonce":33356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":33356},{"type":"v1","addr":"172.21.15.70:6811","nonce":33356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":33356},{"type":"v1","addr":"172.21.15.70:6815","nonce":33356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":33356},{"type":"v1","addr":"172.21.15.70:6813","nonce":33356}]},"public_addr":"172.21.15.70:6809/33356","cluster_addr":"172.21.15.70:6811/33356","heartbeat_back_addr":"172.21.15.70:6815/33356","heartbeat_front_addr":"172.21.15.70:6813/33356","state":["exists","up"]},{"osd":3,"uuid":"6df66ae7-e835-4d8f-bd27-243d7a9705b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":112,"up_from":131,"up_thru":131,"down_at":113,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":58254},{"type":"v1","addr":"172.21.15.70:6825","nonce":58254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":58254},{"type":"v1","addr":"172.21.15.70:6827","nonce":58254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":58254},{"type":"v1","addr":"172.21.15.70:6831","nonce":58254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":58254},{"type":"v1","addr":"172.21.15.70:6829","nonce":58254}]},"public_addr":"172.21.15.70:6825/58254","cluster_addr":"172.21.15.70:6827/58254","heartbeat_back_addr":"172.21.15.70:6831/58254","heartbeat_front_addr":"172.21.15.70:6829/58254","state":["exists","up"]},{"osd":4,"uuid":"29f52d62-d4c3-49f9-992f-23817f5622d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":33331},{"type":"v1","addr":"172.21.15.192:6803","nonce":33331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6805","nonce":33331},{"type":"v1","addr":"172.21.15.192:6807","nonce":33331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6813","nonce":33331},{"type":"v1","addr":"172.21.15.192:6815","nonce":33331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6809","nonce":33331},{"type":"v1","addr":"172.21.15.192:6811","nonce":33331}]},"public_addr":"172.21.15.192:6803/33331","cluster_addr":"172.21.15.192:6807/33331","heartbeat_back_addr":"172.21.15.192:6815/33331","heartbeat_front_addr":"172.21.15.192:6811/33331","state":["exists","up"]},{"osd":5,"uuid":"0c1a275a-3c75-4e0b-bd35-d34b64d99c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":33333},{"type":"v1","addr":"172.21.15.192:6817","nonce":33333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":33333},{"type":"v1","addr":"172.21.15.192:6819","nonce":33333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":33333},{"type":"v1","addr":"172.21.15.192:6823","nonce":33333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":33333},{"type":"v1","addr":"172.21.15.192:6821","nonce":33333}]},"public_addr":"172.21.15.192:6817/33333","cluster_addr":"172.21.15.192:6819/33333","heartbeat_back_addr":"172.21.15.192:6823/33333","heartbeat_front_addr":"172.21.15.192:6821/33333","state":["exists","up"]},{"osd":6,"uuid":"d79f0070-ce58-48e7-a56a-2762ec76a271","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6801","nonce":33332},{"type":"v1","addr":"172.21.15.192:6802","nonce":33332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":33332},{"type":"v1","addr":"172.21.15.192:6806","nonce":33332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":33332},{"type":"v1","addr":"172.21.15.192:6814","nonce":33332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":33332},{"type":"v1","addr":"172.21.15.192:6810","nonce":33332}]},"public_addr":"172.21.15.192:6802/33332","cluster_addr":"172.21.15.192:6806/33332","heartbeat_back_addr":"172.21.15.192:6814/33332","heartbeat_front_addr":"172.21.15.192:6810/33332","state":["exists","up"]},{"osd":7,"uuid":"537fa5df-3d0a-4dd4-8155-20ee94e41c82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":33362},{"type":"v1","addr":"172.21.15.192:6825","nonce":33362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":33362},{"type":"v1","addr":"172.21.15.192:6827","nonce":33362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":33362},{"type":"v1","addr":"172.21.15.192:6831","nonce":33362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":33362},{"type":"v1","addr":"172.21.15.192:6829","nonce":33362}]},"public_addr":"172.21.15.192:6825/33362","cluster_addr":"172.21.15.192:6827/33362","heartbeat_back_addr":"172.21.15.192:6831/33362","heartbeat_front_addr":"172.21.15.192:6829/33362","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.581514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.674271+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.545402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.733565+0000","dead_epoch":113},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.623479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.730066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:10.556650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:43:09.702368+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.8","mappings":[{"from":5,"to":1}]},{"pgid":"4.14","mappings":[{"from":1,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:46:02.416 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T11:46:02.417 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:46:02.546 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:46:02.547 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:46:02.547 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:46:02.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T11:46:02.817 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T11:46:02.830 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T11:46:02.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:46:02.946 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:46:02.946 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:46:02.946 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:46:02.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T11:46:03.217 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T11:46:03.229 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T11:46:03.229 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:46:03.346 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:46:03.346 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:46:03.346 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:46:03.356 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T11:46:03.617 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T11:46:03.630 INFO:tasks.ceph:Scrubbing osd.3 2024-09-16T11:46:03.630 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:46:03.747 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:46:03.747 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-16T11:46:03.747 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:46:03.758 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-16T11:46:04.016 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 3 to deep-scrub 2024-09-16T11:46:04.028 INFO:tasks.ceph:Scrubbing osd.4 2024-09-16T11:46:04.028 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:46:04.145 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:46:04.146 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:46:04.146 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:46:04.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-16T11:46:04.415 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 4 to deep-scrub 2024-09-16T11:46:04.427 INFO:tasks.ceph:Scrubbing osd.5 2024-09-16T11:46:04.428 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:46:04.546 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:46:04.546 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:46:04.546 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:46:04.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-16T11:46:04.816 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 5 to deep-scrub 2024-09-16T11:46:04.828 INFO:tasks.ceph:Scrubbing osd.6 2024-09-16T11:46:04.828 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:46:04.945 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:46:04.945 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:46:04.945 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:46:04.955 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-16T11:46:05.212 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 6 to deep-scrub 2024-09-16T11:46:05.224 INFO:tasks.ceph:Scrubbing osd.7 2024-09-16T11:46:05.225 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:46:05.342 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T11:46:05.342 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:46:05.343 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T11:46:05.352 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-16T11:46:05.618 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 7 to deep-scrub 2024-09-16T11:46:05.630 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:46:05.892 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:46:05.893 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:46:05.904 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-16T11:46:05.403311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590574,"num_objects":741,"num_object_clones":0,"num_object_copies":1482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":731,"num_read":135541,"num_read_kb":642951,"num_write":18165,"num_write_kb":292238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16567,"ondisk_log_size":16567,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445764,"kb_used_data":13896,"kb_used_omap":54,"kb_used_meta":2431689,"kb_avail":752528956,"statfs":{"total":773094113280,"available":770589650944,"internally_reserved":0,"allocated":14229504,"data_stored":8318652,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56156,"internal_metadata":2490049700},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.695955"},"pg_stats":[{"pgid":"4.1f","version":"127'327","reported_seq":6237,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725885+0000","last_change":"2024-09-16T11:45:19.597844+0000","last_active":"2024-09-16T11:45:54.725885+0000","last_peered":"2024-09-16T11:45:54.725885+0000","last_clean":"2024-09-16T11:45:54.725885+0000","last_became_active":"2024-09-16T11:43:31.779428+0000","last_became_peered":"2024-09-16T11:43:31.779428+0000","last_unstale":"2024-09-16T11:45:54.725885+0000","last_undegraded":"2024-09-16T11:45:54.725885+0000","last_fullsized":"2024-09-16T11:45:54.725885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:30.437698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034041099999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6762,"num_read_kb":29123,"num_write":304,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"125'449","reported_seq":5346,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566266+0000","last_change":"2024-09-16T11:45:19.592718+0000","last_active":"2024-09-16T11:45:54.566266+0000","last_peered":"2024-09-16T11:45:54.566266+0000","last_clean":"2024-09-16T11:45:54.566266+0000","last_became_active":"2024-09-16T11:43:31.889706+0000","last_became_peered":"2024-09-16T11:43:31.889706+0000","last_unstale":"2024-09-16T11:45:54.566266+0000","last_undegraded":"2024-09-16T11:45:54.566266+0000","last_fullsized":"2024-09-16T11:45:54.566266+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:25.242116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000654113,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5672,"num_read_kb":36055,"num_write":486,"num_write_kb":9211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"132'213","reported_seq":1539,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725493+0000","last_change":"2024-09-16T11:45:39.661471+0000","last_active":"2024-09-16T11:45:54.725493+0000","last_peered":"2024-09-16T11:45:54.725493+0000","last_clean":"2024-09-16T11:45:54.725493+0000","last_became_active":"2024-09-16T11:45:39.636105+0000","last_became_peered":"2024-09-16T11:45:39.636105+0000","last_unstale":"2024-09-16T11:45:54.725493+0000","last_undegraded":"2024-09-16T11:45:54.725493+0000","last_fullsized":"2024-09-16T11:45:54.725493+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:18:52.260065+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.014051187999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1102,"num_read_kb":9776,"num_write":200,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"127'334","reported_seq":2413,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703281+0000","last_change":"2024-09-16T11:45:19.592554+0000","last_active":"2024-09-16T11:45:53.703281+0000","last_peered":"2024-09-16T11:45:53.703281+0000","last_clean":"2024-09-16T11:45:53.703281+0000","last_became_active":"2024-09-16T11:43:31.773661+0000","last_became_peered":"2024-09-16T11:43:31.773661+0000","last_unstale":"2024-09-16T11:45:53.703281+0000","last_undegraded":"2024-09-16T11:45:53.703281+0000","last_fullsized":"2024-09-16T11:45:53.703281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:36.062796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052992100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":2191,"num_read_kb":14785,"num_write":375,"num_write_kb":7446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"132'380","reported_seq":2524,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703902+0000","last_change":"2024-09-16T11:45:47.464749+0000","last_active":"2024-09-16T11:45:53.703902+0000","last_peered":"2024-09-16T11:45:53.703902+0000","last_clean":"2024-09-16T11:45:53.703902+0000","last_became_active":"2024-09-16T11:45:47.463209+0000","last_became_peered":"2024-09-16T11:45:47.463209+0000","last_unstale":"2024-09-16T11:45:53.703902+0000","last_undegraded":"2024-09-16T11:45:53.703902+0000","last_fullsized":"2024-09-16T11:45:53.703902+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:29.503448+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0033897049999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3246,"num_read_kb":18406,"num_write":318,"num_write_kb":4718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"132'534","reported_seq":3517,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703670+0000","last_change":"2024-09-16T11:45:47.464540+0000","last_active":"2024-09-16T11:45:53.703670+0000","last_peered":"2024-09-16T11:45:53.703670+0000","last_clean":"2024-09-16T11:45:53.703670+0000","last_became_active":"2024-09-16T11:45:47.463199+0000","last_became_peered":"2024-09-16T11:45:47.463199+0000","last_unstale":"2024-09-16T11:45:53.703670+0000","last_undegraded":"2024-09-16T11:45:53.703670+0000","last_fullsized":"2024-09-16T11:45:53.703670+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:44.772848+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.01209723,"stat_sum":{"num_bytes":19,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":4102,"num_read_kb":12532,"num_write":637,"num_write_kb":9975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"124'658","reported_seq":4607,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725383+0000","last_change":"2024-09-16T11:45:39.678403+0000","last_active":"2024-09-16T11:45:54.725383+0000","last_peered":"2024-09-16T11:45:54.725383+0000","last_clean":"2024-09-16T11:45:54.725383+0000","last_became_active":"2024-09-16T11:45:39.638104+0000","last_became_peered":"2024-09-16T11:45:39.638104+0000","last_unstale":"2024-09-16T11:45:54.725383+0000","last_undegraded":"2024-09-16T11:45:54.725383+0000","last_fullsized":"2024-09-16T11:45:54.725383+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:18.828819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069266390000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5632,"num_read_kb":29340,"num_write":868,"num_write_kb":4351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"127'221","reported_seq":1540,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703616+0000","last_change":"2024-09-16T11:45:19.592170+0000","last_active":"2024-09-16T11:45:53.703616+0000","last_peered":"2024-09-16T11:45:53.703616+0000","last_clean":"2024-09-16T11:45:53.703616+0000","last_became_active":"2024-09-16T11:43:31.776951+0000","last_became_peered":"2024-09-16T11:43:31.776951+0000","last_unstale":"2024-09-16T11:45:53.703616+0000","last_undegraded":"2024-09-16T11:45:53.703616+0000","last_fullsized":"2024-09-16T11:45:53.703616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:28:39.026621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052508100000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1522,"num_read_kb":7050,"num_write":185,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"132'525","reported_seq":2341,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725539+0000","last_change":"2024-09-16T11:45:39.723943+0000","last_active":"2024-09-16T11:45:54.725539+0000","last_peered":"2024-09-16T11:45:54.725539+0000","last_clean":"2024-09-16T11:45:54.725539+0000","last_became_active":"2024-09-16T11:45:39.642890+0000","last_became_peered":"2024-09-16T11:45:39.642890+0000","last_unstale":"2024-09-16T11:45:54.725539+0000","last_undegraded":"2024-09-16T11:45:54.725539+0000","last_fullsized":"2024-09-16T11:45:54.725539+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:26.709287+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062520060000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4283,"num_read_kb":14493,"num_write":541,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"132'743","reported_seq":6417,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566264+0000","last_change":"2024-09-16T11:45:39.656713+0000","last_active":"2024-09-16T11:45:54.566264+0000","last_peered":"2024-09-16T11:45:54.566264+0000","last_clean":"2024-09-16T11:45:54.566264+0000","last_became_active":"2024-09-16T11:45:39.640293+0000","last_became_peered":"2024-09-16T11:45:39.640293+0000","last_unstale":"2024-09-16T11:45:54.566264+0000","last_undegraded":"2024-09-16T11:45:54.566264+0000","last_fullsized":"2024-09-16T11:45:54.566264+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:29:01.940843+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074727580000000004,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8298,"num_read_kb":47234,"num_write":833,"num_write_kb":15319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"127'657","reported_seq":2814,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725859+0000","last_change":"2024-09-16T11:45:19.636993+0000","last_active":"2024-09-16T11:45:54.725859+0000","last_peered":"2024-09-16T11:45:54.725859+0000","last_clean":"2024-09-16T11:45:54.725859+0000","last_became_active":"2024-09-16T11:43:31.889629+0000","last_became_peered":"2024-09-16T11:43:31.889629+0000","last_unstale":"2024-09-16T11:45:54.725859+0000","last_undegraded":"2024-09-16T11:45:54.725859+0000","last_fullsized":"2024-09-16T11:45:54.725859+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:53.124562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030057499999999997,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":3982,"num_read_kb":26490,"num_write":811,"num_write_kb":16305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"132'665","reported_seq":3461,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725390+0000","last_change":"2024-09-16T11:45:39.707640+0000","last_active":"2024-09-16T11:45:54.725390+0000","last_peered":"2024-09-16T11:45:54.725390+0000","last_clean":"2024-09-16T11:45:54.725390+0000","last_became_active":"2024-09-16T11:45:39.641187+0000","last_became_peered":"2024-09-16T11:45:39.641187+0000","last_unstale":"2024-09-16T11:45:54.725390+0000","last_undegraded":"2024-09-16T11:45:54.725390+0000","last_fullsized":"2024-09-16T11:45:54.725390+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:47:39.464422+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074563379999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4363,"num_read_kb":20582,"num_write":914,"num_write_kb":10744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":3197941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"127'546","reported_seq":2511,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703790+0000","last_change":"2024-09-16T11:45:19.592153+0000","last_active":"2024-09-16T11:45:53.703790+0000","last_peered":"2024-09-16T11:45:53.703790+0000","last_clean":"2024-09-16T11:45:53.703790+0000","last_became_active":"2024-09-16T11:43:31.776781+0000","last_became_peered":"2024-09-16T11:43:31.776781+0000","last_unstale":"2024-09-16T11:45:53.703790+0000","last_undegraded":"2024-09-16T11:45:53.703790+0000","last_fullsized":"2024-09-16T11:45:53.703790+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:14.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032933499999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3911,"num_read_kb":15179,"num_write":595,"num_write_kb":10620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"121'294","reported_seq":1908,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703771+0000","last_change":"2024-09-16T11:45:19.642323+0000","last_active":"2024-09-16T11:45:53.703771+0000","last_peered":"2024-09-16T11:45:53.703771+0000","last_clean":"2024-09-16T11:45:53.703771+0000","last_became_active":"2024-09-16T11:43:31.780164+0000","last_became_peered":"2024-09-16T11:43:31.780164+0000","last_unstale":"2024-09-16T11:45:53.703771+0000","last_undegraded":"2024-09-16T11:45:53.703771+0000","last_fullsized":"2024-09-16T11:45:53.703771+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:56.846891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079981,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1687,"num_read_kb":10324,"num_write":299,"num_write_kb":6058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"132'804","reported_seq":2260,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725294+0000","last_change":"2024-09-16T11:45:39.712869+0000","last_active":"2024-09-16T11:45:54.725294+0000","last_peered":"2024-09-16T11:45:54.725294+0000","last_clean":"2024-09-16T11:45:54.725294+0000","last_became_active":"2024-09-16T11:45:39.641126+0000","last_became_peered":"2024-09-16T11:45:39.641126+0000","last_unstale":"2024-09-16T11:45:54.725294+0000","last_undegraded":"2024-09-16T11:45:54.725294+0000","last_fullsized":"2024-09-16T11:45:54.725294+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:23.732386+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074597470000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":4259,"num_read_kb":18254,"num_write":1055,"num_write_kb":13061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"132'401","reported_seq":1908,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725271+0000","last_change":"2024-09-16T11:45:39.666850+0000","last_active":"2024-09-16T11:45:54.725271+0000","last_peered":"2024-09-16T11:45:54.725271+0000","last_clean":"2024-09-16T11:45:54.725271+0000","last_became_active":"2024-09-16T11:45:39.637293+0000","last_became_peered":"2024-09-16T11:45:39.637293+0000","last_unstale":"2024-09-16T11:45:54.725271+0000","last_undegraded":"2024-09-16T11:45:54.725271+0000","last_fullsized":"2024-09-16T11:45:54.725271+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:39.210719+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.016205368000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3249,"num_read_kb":9637,"num_write":346,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725801+0000","last_change":"2024-09-16T11:43:16.733061+0000","last_active":"2024-09-16T11:45:54.725801+0000","last_peered":"2024-09-16T11:45:54.725801+0000","last_clean":"2024-09-16T11:45:54.725801+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:45:54.725801+0000","last_undegraded":"2024-09-16T11:45:54.725801+0000","last_fullsized":"2024-09-16T11:45:54.725801+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:50.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"125'347","reported_seq":1761,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703661+0000","last_change":"2024-09-16T11:45:47.465149+0000","last_active":"2024-09-16T11:45:53.703661+0000","last_peered":"2024-09-16T11:45:53.703661+0000","last_clean":"2024-09-16T11:45:53.703661+0000","last_became_active":"2024-09-16T11:45:47.464115+0000","last_became_peered":"2024-09-16T11:45:47.464115+0000","last_unstale":"2024-09-16T11:45:53.703661+0000","last_undegraded":"2024-09-16T11:45:53.703661+0000","last_fullsized":"2024-09-16T11:45:53.703661+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:34.356992+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00038392500000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2491,"num_read_kb":9150,"num_write":283,"num_write_kb":9998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703962+0000","last_change":"2024-09-16T11:43:16.741003+0000","last_active":"2024-09-16T11:45:53.703962+0000","last_peered":"2024-09-16T11:45:53.703962+0000","last_clean":"2024-09-16T11:45:53.703962+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:45:53.703962+0000","last_undegraded":"2024-09-16T11:45:53.703962+0000","last_fullsized":"2024-09-16T11:45:53.703962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:37.048584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"132'477","reported_seq":4581,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725271+0000","last_change":"2024-09-16T11:45:39.723814+0000","last_active":"2024-09-16T11:45:54.725271+0000","last_peered":"2024-09-16T11:45:54.725271+0000","last_clean":"2024-09-16T11:45:54.725271+0000","last_became_active":"2024-09-16T11:45:39.641298+0000","last_became_peered":"2024-09-16T11:45:39.641298+0000","last_unstale":"2024-09-16T11:45:54.725271+0000","last_undegraded":"2024-09-16T11:45:54.725271+0000","last_fullsized":"2024-09-16T11:45:54.725271+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:57.805852+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0094940819999999992,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5544,"num_read_kb":23694,"num_write":515,"num_write_kb":8223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.722985+0000","last_change":"2024-09-16T11:43:16.739114+0000","last_active":"2024-09-16T11:45:53.722985+0000","last_peered":"2024-09-16T11:45:53.722985+0000","last_clean":"2024-09-16T11:45:53.722985+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:45:53.722985+0000","last_undegraded":"2024-09-16T11:45:53.722985+0000","last_fullsized":"2024-09-16T11:45:53.722985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:11.365261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"127'627","reported_seq":6867,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566281+0000","last_change":"2024-09-16T11:45:19.593742+0000","last_active":"2024-09-16T11:45:54.566281+0000","last_peered":"2024-09-16T11:45:54.566281+0000","last_clean":"2024-09-16T11:45:54.566281+0000","last_became_active":"2024-09-16T11:43:31.777896+0000","last_became_peered":"2024-09-16T11:43:31.777896+0000","last_unstale":"2024-09-16T11:45:54.566281+0000","last_undegraded":"2024-09-16T11:45:54.566281+0000","last_fullsized":"2024-09-16T11:45:54.566281+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:35.052654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011770719999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":7410,"num_read_kb":47228,"num_write":617,"num_write_kb":8894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704146+0000","last_change":"2024-09-16T11:43:16.741012+0000","last_active":"2024-09-16T11:45:53.704146+0000","last_peered":"2024-09-16T11:45:53.704146+0000","last_clean":"2024-09-16T11:45:53.704146+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:45:53.704146+0000","last_undegraded":"2024-09-16T11:45:53.704146+0000","last_fullsized":"2024-09-16T11:45:53.704146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:42.550952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"126'382","reported_seq":1083,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703876+0000","last_change":"2024-09-16T11:45:19.642384+0000","last_active":"2024-09-16T11:45:53.703876+0000","last_peered":"2024-09-16T11:45:53.703876+0000","last_clean":"2024-09-16T11:45:53.703876+0000","last_became_active":"2024-09-16T11:43:31.778506+0000","last_became_peered":"2024-09-16T11:43:31.778506+0000","last_unstale":"2024-09-16T11:45:53.703876+0000","last_undegraded":"2024-09-16T11:45:53.703876+0000","last_fullsized":"2024-09-16T11:45:53.703876+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:12.232493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011478079999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1438,"num_read_kb":4478,"num_write":491,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":110,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703900+0000","last_change":"2024-09-16T11:45:47.462953+0000","last_active":"2024-09-16T11:45:53.703900+0000","last_peered":"2024-09-16T11:45:53.703900+0000","last_clean":"2024-09-16T11:45:53.703900+0000","last_became_active":"2024-09-16T11:45:47.462295+0000","last_became_peered":"2024-09-16T11:45:47.462295+0000","last_unstale":"2024-09-16T11:45:53.703900+0000","last_undegraded":"2024-09-16T11:45:53.703900+0000","last_fullsized":"2024-09-16T11:45:53.703900+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:57.786498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":140,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725763+0000","last_change":"2024-09-16T11:43:16.739428+0000","last_active":"2024-09-16T11:45:54.725763+0000","last_peered":"2024-09-16T11:45:54.725763+0000","last_clean":"2024-09-16T11:45:54.725763+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:45:54.725763+0000","last_undegraded":"2024-09-16T11:45:54.725763+0000","last_fullsized":"2024-09-16T11:45:54.725763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.862182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"127'734","reported_seq":2603,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704134+0000","last_change":"2024-09-16T11:45:19.641341+0000","last_active":"2024-09-16T11:45:53.704134+0000","last_peered":"2024-09-16T11:45:53.704134+0000","last_clean":"2024-09-16T11:45:53.704134+0000","last_became_active":"2024-09-16T11:43:31.777796+0000","last_became_peered":"2024-09-16T11:43:31.777796+0000","last_unstale":"2024-09-16T11:45:53.704134+0000","last_undegraded":"2024-09-16T11:45:53.704134+0000","last_fullsized":"2024-09-16T11:45:53.704134+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:33.410999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051098400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4214,"num_read_kb":15199,"num_write":913,"num_write_kb":10139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":132,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704039+0000","last_change":"2024-09-16T11:43:22.440136+0000","last_active":"2024-09-16T11:45:53.704039+0000","last_peered":"2024-09-16T11:45:53.704039+0000","last_clean":"2024-09-16T11:45:53.704039+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:45:53.704039+0000","last_undegraded":"2024-09-16T11:45:53.704039+0000","last_fullsized":"2024-09-16T11:45:53.704039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:18.461948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703187+0000","last_change":"2024-09-16T11:45:39.635580+0000","last_active":"2024-09-16T11:45:53.703187+0000","last_peered":"2024-09-16T11:45:53.703187+0000","last_clean":"2024-09-16T11:45:53.703187+0000","last_became_active":"2024-09-16T11:45:39.635334+0000","last_became_peered":"2024-09-16T11:45:39.635334+0000","last_unstale":"2024-09-16T11:45:53.703187+0000","last_undegraded":"2024-09-16T11:45:53.703187+0000","last_fullsized":"2024-09-16T11:45:53.703187+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:43.033024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"125'546","reported_seq":3390,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.704084+0000","last_change":"2024-09-16T11:45:19.641909+0000","last_active":"2024-09-16T11:45:53.704084+0000","last_peered":"2024-09-16T11:45:53.704084+0000","last_clean":"2024-09-16T11:45:53.704084+0000","last_became_active":"2024-09-16T11:43:31.777967+0000","last_became_peered":"2024-09-16T11:43:31.777967+0000","last_unstale":"2024-09-16T11:45:53.704084+0000","last_undegraded":"2024-09-16T11:45:53.704084+0000","last_fullsized":"2024-09-16T11:45:53.704084+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:43.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081483900000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3645,"num_read_kb":23388,"num_write":630,"num_write_kb":9618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"112'2","reported_seq":133,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566166+0000","last_change":"2024-09-16T11:43:22.437965+0000","last_active":"2024-09-16T11:45:54.566166+0000","last_peered":"2024-09-16T11:45:54.566166+0000","last_clean":"2024-09-16T11:45:54.566166+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:45:54.566166+0000","last_undegraded":"2024-09-16T11:45:54.566166+0000","last_fullsized":"2024-09-16T11:45:54.566166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:52.884360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.723066+0000","last_change":"2024-09-16T11:43:16.739059+0000","last_active":"2024-09-16T11:45:53.723066+0000","last_peered":"2024-09-16T11:45:53.723066+0000","last_clean":"2024-09-16T11:45:53.723066+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:45:53.723066+0000","last_undegraded":"2024-09-16T11:45:53.723066+0000","last_fullsized":"2024-09-16T11:45:53.723066+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.841579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"127'407","reported_seq":1770,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566334+0000","last_change":"2024-09-16T11:45:19.592986+0000","last_active":"2024-09-16T11:45:54.566334+0000","last_peered":"2024-09-16T11:45:54.566334+0000","last_clean":"2024-09-16T11:45:54.566334+0000","last_became_active":"2024-09-16T11:43:31.776645+0000","last_became_peered":"2024-09-16T11:43:31.776645+0000","last_unstale":"2024-09-16T11:45:54.566334+0000","last_undegraded":"2024-09-16T11:45:54.566334+0000","last_fullsized":"2024-09-16T11:45:54.566334+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:53.198323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066659399999999995,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1650,"num_read_kb":12114,"num_write":331,"num_write_kb":15245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"84'3","reported_seq":97,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725388+0000","last_change":"2024-09-16T11:45:39.638395+0000","last_active":"2024-09-16T11:45:54.725388+0000","last_peered":"2024-09-16T11:45:54.725388+0000","last_clean":"2024-09-16T11:45:54.725388+0000","last_became_active":"2024-09-16T11:45:39.637629+0000","last_became_peered":"2024-09-16T11:45:39.637629+0000","last_unstale":"2024-09-16T11:45:54.725388+0000","last_undegraded":"2024-09-16T11:45:54.725388+0000","last_fullsized":"2024-09-16T11:45:54.725388+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:21.431624+0000","last_clean_scrub_stamp":"2024-09-16T11:43:21.431624+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:56.174857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":139,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725989+0000","last_change":"2024-09-16T11:43:16.738779+0000","last_active":"2024-09-16T11:45:54.725989+0000","last_peered":"2024-09-16T11:45:54.725989+0000","last_clean":"2024-09-16T11:45:54.725989+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:45:54.725989+0000","last_undegraded":"2024-09-16T11:45:54.725989+0000","last_fullsized":"2024-09-16T11:45:54.725989+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:13.696451+0000","last_clean_scrub_stamp":"2024-09-16T11:43:13.696451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:00.174284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":226,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.723073+0000","last_change":"2024-09-16T11:43:12.702599+0000","last_active":"2024-09-16T11:45:53.723073+0000","last_peered":"2024-09-16T11:45:53.723073+0000","last_clean":"2024-09-16T11:45:53.723073+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:45:53.723073+0000","last_undegraded":"2024-09-16T11:45:53.723073+0000","last_fullsized":"2024-09-16T11:45:53.723073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:11.689589+0000","last_clean_scrub_stamp":"2024-09-16T11:43:11.689589+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:10.687865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"125'523","reported_seq":3438,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725834+0000","last_change":"2024-09-16T11:45:19.598172+0000","last_active":"2024-09-16T11:45:54.725834+0000","last_peered":"2024-09-16T11:45:54.725834+0000","last_clean":"2024-09-16T11:45:54.725834+0000","last_became_active":"2024-09-16T11:43:31.782071+0000","last_became_peered":"2024-09-16T11:43:31.782071+0000","last_unstale":"2024-09-16T11:45:54.725834+0000","last_undegraded":"2024-09-16T11:45:54.725834+0000","last_fullsized":"2024-09-16T11:45:54.725834+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:15.516215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000599604,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3801,"num_read_kb":24361,"num_write":540,"num_write_kb":7978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"127'665","reported_seq":9006,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703769+0000","last_change":"2024-09-16T11:45:19.641414+0000","last_active":"2024-09-16T11:45:53.703769+0000","last_peered":"2024-09-16T11:45:53.703769+0000","last_clean":"2024-09-16T11:45:53.703769+0000","last_became_active":"2024-09-16T11:44:10.920131+0000","last_became_peered":"2024-09-16T11:44:10.920131+0000","last_unstale":"2024-09-16T11:45:53.703769+0000","last_undegraded":"2024-09-16T11:45:53.703769+0000","last_fullsized":"2024-09-16T11:45:53.703769+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:55:10.158414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082161099999999998,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":9498,"num_read_kb":69976,"num_write":712,"num_write_kb":19149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":8960357,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"127'878","reported_seq":3861,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703624+0000","last_change":"2024-09-16T11:45:47.465826+0000","last_active":"2024-09-16T11:45:53.703624+0000","last_peered":"2024-09-16T11:45:53.703624+0000","last_clean":"2024-09-16T11:45:53.703624+0000","last_became_active":"2024-09-16T11:45:47.464353+0000","last_became_peered":"2024-09-16T11:45:47.464353+0000","last_unstale":"2024-09-16T11:45:53.703624+0000","last_undegraded":"2024-09-16T11:45:53.703624+0000","last_fullsized":"2024-09-16T11:45:53.703624+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:24.438451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001068741,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":8005,"num_read_kb":12353,"num_write":894,"num_write_kb":6127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"127'623","reported_seq":1817,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.726002+0000","last_change":"2024-09-16T11:45:19.598161+0000","last_active":"2024-09-16T11:45:54.726002+0000","last_peered":"2024-09-16T11:45:54.726002+0000","last_clean":"2024-09-16T11:45:54.726002+0000","last_became_active":"2024-09-16T11:43:31.779384+0000","last_became_peered":"2024-09-16T11:43:31.779384+0000","last_unstale":"2024-09-16T11:45:54.726002+0000","last_undegraded":"2024-09-16T11:45:54.726002+0000","last_fullsized":"2024-09-16T11:45:54.726002+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:34.933146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000647592,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":2523,"num_read_kb":9199,"num_write":666,"num_write_kb":6203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"125'377","reported_seq":3380,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.566193+0000","last_change":"2024-09-16T11:45:19.593453+0000","last_active":"2024-09-16T11:45:54.566193+0000","last_peered":"2024-09-16T11:45:54.566193+0000","last_clean":"2024-09-16T11:45:54.566193+0000","last_became_active":"2024-09-16T11:43:31.775335+0000","last_became_peered":"2024-09-16T11:43:31.775335+0000","last_unstale":"2024-09-16T11:45:54.566193+0000","last_undegraded":"2024-09-16T11:45:54.566193+0000","last_fullsized":"2024-09-16T11:45:54.566193+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:07.521713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011352389999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4122,"num_read_kb":15191,"num_write":417,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"132'521","reported_seq":2268,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703740+0000","last_change":"2024-09-16T11:45:47.465501+0000","last_active":"2024-09-16T11:45:53.703740+0000","last_peered":"2024-09-16T11:45:53.703740+0000","last_clean":"2024-09-16T11:45:53.703740+0000","last_became_active":"2024-09-16T11:45:47.463487+0000","last_became_peered":"2024-09-16T11:45:47.463487+0000","last_unstale":"2024-09-16T11:45:53.703740+0000","last_undegraded":"2024-09-16T11:45:53.703740+0000","last_fullsized":"2024-09-16T11:45:53.703740+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:41.597993+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001754511,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4037,"num_read_kb":5871,"num_write":454,"num_write_kb":2926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"125'616","reported_seq":5073,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703823+0000","last_change":"2024-09-16T11:45:47.462731+0000","last_active":"2024-09-16T11:45:53.703823+0000","last_peered":"2024-09-16T11:45:53.703823+0000","last_clean":"2024-09-16T11:45:53.703823+0000","last_became_active":"2024-09-16T11:45:47.461210+0000","last_became_peered":"2024-09-16T11:45:47.461210+0000","last_unstale":"2024-09-16T11:45:53.703823+0000","last_undegraded":"2024-09-16T11:45:53.703823+0000","last_fullsized":"2024-09-16T11:45:53.703823+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:17.838544+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00043040400000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6463,"num_read_kb":36386,"num_write":811,"num_write_kb":14056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"126'605","reported_seq":1931,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703756+0000","last_change":"2024-09-16T11:45:47.465926+0000","last_active":"2024-09-16T11:45:53.703756+0000","last_peered":"2024-09-16T11:45:53.703756+0000","last_clean":"2024-09-16T11:45:53.703756+0000","last_became_active":"2024-09-16T11:45:47.464052+0000","last_became_peered":"2024-09-16T11:45:47.464052+0000","last_unstale":"2024-09-16T11:45:53.703756+0000","last_undegraded":"2024-09-16T11:45:53.703756+0000","last_fullsized":"2024-09-16T11:45:53.703756+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:40.083173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059308799999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2764,"num_read_kb":9887,"num_write":657,"num_write_kb":5333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"127'408","reported_seq":2117,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T11:45:53.703980+0000","last_change":"2024-09-16T11:45:39.667068+0000","last_active":"2024-09-16T11:45:53.703980+0000","last_peered":"2024-09-16T11:45:53.703980+0000","last_clean":"2024-09-16T11:45:53.703980+0000","last_became_active":"2024-09-16T11:45:39.643207+0000","last_became_peered":"2024-09-16T11:45:39.643207+0000","last_unstale":"2024-09-16T11:45:53.703980+0000","last_undegraded":"2024-09-16T11:45:53.703980+0000","last_fullsized":"2024-09-16T11:45:53.703980+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:12.700226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070987940000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3609,"num_read_kb":5161,"num_write":362,"num_write_kb":8941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":728,"num_read":135475,"num_read_kb":642896,"num_write":18060,"num_write_kb":290606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},"log_size":16487,"ondisk_log_size":16487,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542182,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299128,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072712,"statfs":{"total":96636764160,"available":96330457088,"internally_reserved":0,"allocated":1814528,"data_stored":1481880,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.441,"15min":0.443},"min":{"1min":0.19600000000000001,"5min":0.182,"15min":0.182},"max":{"1min":0.61299999999999999,"5min":0.628,"15min":0.628},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.61799999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.247,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":4.3559999999999999,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.435,"15min":0.435},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.63300000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.28000000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.68400000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":1.532,"15min":1.532},"last":0.51700000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.5640000000000001,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":0.495}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.26000000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.67500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.45400000000000001,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.30099999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.437,"15min":0.44},"min":{"1min":0.27600000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.65200000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48099999999999998,"15min":0.48499999999999999},"min":{"1min":0.318,"5min":0.251,"15min":0.251},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.34399999999999997}]},{"osd":6,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47299999999999998,"15min":0.47099999999999997},"min":{"1min":0.377,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72799999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66200000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.38300000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315728,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94056112,"statfs":{"total":96636764160,"available":96313458688,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.438,"15min":0.44400000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52800000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58999999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.66800000000000004}]},{"osd":1,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.33800000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.53100000000000003,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.439,"15min":0.442},"min":{"1min":0.27400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.61799999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.55500000000000005}]},{"osd":2,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.30399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.47899999999999998,"15min":0.48099999999999998},"min":{"1min":0.35899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.63,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.45500000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.45000000000000001,"15min":0.45800000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64300000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.41699999999999998}]},{"osd":7,"last update":"Mon Sep 16 11:45:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.48699999999999999,"15min":0.498},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.502,"15min":0.51100000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.503}]}]},{"osd":3,"up_from":131,"seq":562640715783,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302168,"kb_used_data":1728,"kb_used_omap":43,"kb_used_meta":300372,"kb_avail":94069672,"statfs":{"total":96636764160,"available":96327344128,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44228,"internal_metadata":307581756},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315724,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94056116,"statfs":{"total":96636764160,"available":96313462784,"internally_reserved":0,"allocated":1765376,"data_stored":892439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":1.087,"5min":0.495,"15min":0.39700000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":11.218,"5min":11.218,"15min":11.218},"last":0.26800000000000002}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32200000000000001},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.311},{"interface":"front","average":{"1min":0.372,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.29899999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.36199999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72399999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.46600000000000003}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53500000000000003,"15min":0.54200000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.47799999999999998}]},{"osd":6,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68600000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.441}]},{"osd":7,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.871,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.41899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311556,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94060284,"statfs":{"total":96636764160,"available":96317730816,"internally_reserved":0,"allocated":1757184,"data_stored":891472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.22700000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.59899999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.42999999999999999,"15min":0.438},"min":{"1min":0.22900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.61099999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.52200000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.45300000000000001,"15min":0.45700000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.75800000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.48399999999999999,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":1.026,"15min":1.026},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.47499999999999998,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79600000000000004,"5min":1.01,"15min":1.01},"last":0.53500000000000003}]},{"osd":3,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.50700000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39800000000000002,"15min":0.40300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54700000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.27300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40999999999999998}]},{"osd":5,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.45300000000000001,"15min":0.46500000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.755,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.316},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.42099999999999999,"15min":0.433},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52400000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35099999999999998}]},{"osd":7,"last update":"Mon Sep 16 11:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.48199999999999998,"15min":0.495},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67100000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.372}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303244,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068596,"statfs":{"total":96636764160,"available":96326242304,"internally_reserved":0,"allocated":1769472,"data_stored":892522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.47599999999999998,"15min":0.441},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":5.8399999999999999,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.44900000000000001,"15min":0.45100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48799999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.60699999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.42499999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.48599999999999999,"15min":0.496},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.621,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.49099999999999999,"15min":0.5},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.53900000000000003}]},{"osd":3,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.59299999999999997}]},{"osd":5,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.42099999999999999,"15min":0.435},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.44700000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.45500000000000002,"15min":0.46899999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57699999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.44}]},{"osd":6,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.505,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.443,"5min":0.53500000000000003,"15min":0.55100000000000005},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60299999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61499999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56599999999999995,"5min":0.873,"15min":0.873},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58999999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.52800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299076,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072764,"statfs":{"total":96636764160,"available":96330510336,"internally_reserved":0,"allocated":1761280,"data_stored":892420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.26200000000000001,"5min":0.29399999999999998,"15min":0.29999999999999999},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.46500000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.32800000000000001,"15min":0.32500000000000001},"min":{"1min":0.215,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.246}]},{"osd":1,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32900000000000001,"15min":0.33200000000000002},"min":{"1min":0.23000000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.437,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34799999999999998,"15min":0.34899999999999998},"min":{"1min":0.255,"5min":0.192,"15min":0.192},"max":{"1min":0.53600000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.375}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.48299999999999998}]},{"osd":4,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.52700000000000002,"15min":0.54100000000000004},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.35599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.746}]},{"osd":6,"last update":"Mon Sep 16 11:45:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57999999999999996,"15min":0.58899999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.57799999999999996,"15min":0.58799999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.55300000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.59199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299140,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072700,"statfs":{"total":96636764160,"available":96330444800,"internally_reserved":0,"allocated":1822720,"data_stored":1482871,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.39600000000000002,"15min":0.33800000000000002},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":8.5050000000000008,"5min":8.5050000000000008,"15min":8.5050000000000008},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32100000000000001},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.436,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.48599999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.39600000000000002,"15min":0.33600000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":8.4689999999999994,"5min":8.4689999999999994,"15min":8.4689999999999994},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.23699999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47699999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.41499999999999998}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.432},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.40000000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.57399999999999995,"15min":0.51400000000000001},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":8.5299999999999994,"5min":8.5299999999999994,"15min":8.5299999999999994},"last":0.628},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.503,"15min":0.505},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71199999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.63900000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.65100000000000002,"15min":0.60299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":8.6039999999999992,"5min":8.6039999999999992,"15min":8.6039999999999992},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.55300000000000005,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69099999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.59699999999999998}]},{"osd":6,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.64200000000000002,"5min":1.075,"15min":1.075},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.67500000000000004,"15min":0.628},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":8.5809999999999995,"5min":8.5809999999999995,"15min":8.5809999999999995},"last":0.66400000000000003}]},{"osd":7,"last update":"Mon Sep 16 11:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.64800000000000002,"5min":0.747,"15min":0.747},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.61199999999999999,"15min":0.55200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":8.5570000000000004,"5min":8.5570000000000004,"15min":8.5570000000000004},"last":0.71399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:46:05.910 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.910 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.910 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.910 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.910 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.911 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.911 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.911 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.911 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.911 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.911 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.911 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.911 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.911 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.912 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.912 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.912 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T11:43:13.696451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=13, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.912 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.912 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T11:43:13.696451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=13, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.912 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.912 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T11:43:13.696451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=13, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.912 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.912 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-16T11:43:13.696451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=13, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.913 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.913 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-16T11:43:21.431624+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=21, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.913 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-16T11:43:13.696451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=13, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.913 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.913 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-16T11:43:21.431624+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=21, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.913 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-16T11:43:13.696451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=13, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.913 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.913 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-16T11:43:21.431624+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=21, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.913 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T11:43:13.696451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=13, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.914 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.914 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-16T11:43:21.431624+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=21, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.914 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T11:43:13.696451+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=13, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.914 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T11:43:11.689589+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=11, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.914 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.914 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.914 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.914 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.914 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.915 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.915 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.915 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.915 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:05.915 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T11:46:25.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:46:26.192 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:46:26.193 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:46:26.204 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-09-16T11:46:25.408290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590657,"num_objects":742,"num_object_clones":0,"num_object_copies":1484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":731,"num_read":135541,"num_read_kb":642951,"num_write":18166,"num_write_kb":292239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":251,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17029,"ondisk_log_size":17029,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2412432,"kb_used_data":13904,"kb_used_omap":54,"kb_used_meta":2398409,"kb_avail":752562288,"statfs":{"total":773094113280,"available":770623782912,"internally_reserved":0,"allocated":14237696,"data_stored":8318818,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56156,"internal_metadata":2455970980},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":83,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003161"},"pg_stats":[{"pgid":"4.1f","version":"147'340","reported_seq":6261,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:17.850091+0000","last_change":"2024-09-16T11:46:17.850036+0000","last_active":"2024-09-16T11:46:17.850091+0000","last_peered":"2024-09-16T11:46:17.850091+0000","last_clean":"2024-09-16T11:46:17.850091+0000","last_became_active":"2024-09-16T11:43:31.779428+0000","last_became_peered":"2024-09-16T11:43:31.779428+0000","last_unstale":"2024-09-16T11:46:17.850091+0000","last_undegraded":"2024-09-16T11:46:17.850091+0000","last_fullsized":"2024-09-16T11:46:17.850091+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'340","last_scrub_stamp":"2024-09-16T11:46:17.849933+0000","last_deep_scrub":"147'340","last_deep_scrub_stamp":"2024-09-16T11:46:17.849933+0000","last_clean_scrub_stamp":"2024-09-16T11:46:17.849933+0000","objects_scrubbed":13,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:04.745251+0000","scrub_duration":0.0093597230000000003,"objects_trimmed":0,"snaptrim_duration":0.00034041099999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6762,"num_read_kb":29123,"num_write":304,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"125'449","reported_seq":5347,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.709117+0000","last_change":"2024-09-16T11:45:19.592718+0000","last_active":"2024-09-16T11:45:54.709117+0000","last_peered":"2024-09-16T11:45:54.709117+0000","last_clean":"2024-09-16T11:45:54.709117+0000","last_became_active":"2024-09-16T11:43:31.889706+0000","last_became_peered":"2024-09-16T11:43:31.889706+0000","last_unstale":"2024-09-16T11:45:54.709117+0000","last_undegraded":"2024-09-16T11:45:54.709117+0000","last_fullsized":"2024-09-16T11:45:54.709117+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000654113,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5672,"num_read_kb":36055,"num_write":486,"num_write_kb":9211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"132'213","reported_seq":1540,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.962022+0000","last_change":"2024-09-16T11:45:39.661471+0000","last_active":"2024-09-16T11:45:54.962022+0000","last_peered":"2024-09-16T11:45:54.962022+0000","last_clean":"2024-09-16T11:45:54.962022+0000","last_became_active":"2024-09-16T11:45:39.636105+0000","last_became_peered":"2024-09-16T11:45:39.636105+0000","last_unstale":"2024-09-16T11:45:54.962022+0000","last_undegraded":"2024-09-16T11:45:54.962022+0000","last_fullsized":"2024-09-16T11:45:54.962022+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.014051187999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1102,"num_read_kb":9776,"num_write":200,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"147'349","reported_seq":2439,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:05.481663+0000","last_change":"2024-09-16T11:46:05.481642+0000","last_active":"2024-09-16T11:46:05.481663+0000","last_peered":"2024-09-16T11:46:05.481663+0000","last_clean":"2024-09-16T11:46:05.481663+0000","last_became_active":"2024-09-16T11:43:31.773661+0000","last_became_peered":"2024-09-16T11:43:31.773661+0000","last_unstale":"2024-09-16T11:46:05.481663+0000","last_undegraded":"2024-09-16T11:46:05.481663+0000","last_fullsized":"2024-09-16T11:46:05.481663+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'349","last_scrub_stamp":"2024-09-16T11:46:05.481572+0000","last_deep_scrub":"147'349","last_deep_scrub_stamp":"2024-09-16T11:46:05.481572+0000","last_clean_scrub_stamp":"2024-09-16T11:46:05.481572+0000","objects_scrubbed":15,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:28:35.138063+0000","scrub_duration":0.0088023839999999999,"objects_trimmed":0,"snaptrim_duration":0.00052992100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":2191,"num_read_kb":14785,"num_write":375,"num_write_kb":7446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"132'380","reported_seq":2525,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.726132+0000","last_change":"2024-09-16T11:45:47.464749+0000","last_active":"2024-09-16T11:45:54.726132+0000","last_peered":"2024-09-16T11:45:54.726132+0000","last_clean":"2024-09-16T11:45:54.726132+0000","last_became_active":"2024-09-16T11:45:47.463209+0000","last_became_peered":"2024-09-16T11:45:47.463209+0000","last_unstale":"2024-09-16T11:45:54.726132+0000","last_undegraded":"2024-09-16T11:45:54.726132+0000","last_fullsized":"2024-09-16T11:45:54.726132+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0033897049999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3246,"num_read_kb":18406,"num_write":318,"num_write_kb":4718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"147'566","reported_seq":3561,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:18.137237+0000","last_change":"2024-09-16T11:46:18.137185+0000","last_active":"2024-09-16T11:46:18.137237+0000","last_peered":"2024-09-16T11:46:18.137237+0000","last_clean":"2024-09-16T11:46:18.137237+0000","last_became_active":"2024-09-16T11:45:47.463199+0000","last_became_peered":"2024-09-16T11:45:47.463199+0000","last_unstale":"2024-09-16T11:46:18.137237+0000","last_undegraded":"2024-09-16T11:46:18.137237+0000","last_fullsized":"2024-09-16T11:46:18.137237+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'566","last_scrub_stamp":"2024-09-16T11:46:18.137055+0000","last_deep_scrub":"147'566","last_deep_scrub_stamp":"2024-09-16T11:46:18.137055+0000","last_clean_scrub_stamp":"2024-09-16T11:46:18.137055+0000","objects_scrubbed":32,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:44.286191+0000","scrub_duration":0.027672068000000001,"objects_trimmed":12,"snaptrim_duration":0.01209723,"stat_sum":{"num_bytes":19,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":4102,"num_read_kb":12532,"num_write":637,"num_write_kb":9975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"124'658","reported_seq":4608,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.961970+0000","last_change":"2024-09-16T11:45:39.678403+0000","last_active":"2024-09-16T11:45:54.961970+0000","last_peered":"2024-09-16T11:45:54.961970+0000","last_clean":"2024-09-16T11:45:54.961970+0000","last_became_active":"2024-09-16T11:45:39.638104+0000","last_became_peered":"2024-09-16T11:45:39.638104+0000","last_unstale":"2024-09-16T11:45:54.961970+0000","last_undegraded":"2024-09-16T11:45:54.961970+0000","last_fullsized":"2024-09-16T11:45:54.961970+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069266390000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5632,"num_read_kb":29340,"num_write":868,"num_write_kb":4351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"147'240","reported_seq":1570,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:09.137966+0000","last_change":"2024-09-16T11:46:09.137936+0000","last_active":"2024-09-16T11:46:09.137966+0000","last_peered":"2024-09-16T11:46:09.137966+0000","last_clean":"2024-09-16T11:46:09.137966+0000","last_became_active":"2024-09-16T11:43:31.776951+0000","last_became_peered":"2024-09-16T11:43:31.776951+0000","last_unstale":"2024-09-16T11:46:09.137966+0000","last_undegraded":"2024-09-16T11:46:09.137966+0000","last_fullsized":"2024-09-16T11:46:09.137966+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'240","last_scrub_stamp":"2024-09-16T11:46:09.137849+0000","last_deep_scrub":"147'240","last_deep_scrub_stamp":"2024-09-16T11:46:09.137849+0000","last_clean_scrub_stamp":"2024-09-16T11:46:09.137849+0000","objects_scrubbed":19,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:00.683567+0000","scrub_duration":0.012833168000000001,"objects_trimmed":0,"snaptrim_duration":0.00052508100000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1522,"num_read_kb":7050,"num_write":185,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"147'547","reported_seq":2374,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:07.379139+0000","last_change":"2024-09-16T11:46:07.379106+0000","last_active":"2024-09-16T11:46:07.379139+0000","last_peered":"2024-09-16T11:46:07.379139+0000","last_clean":"2024-09-16T11:46:07.379139+0000","last_became_active":"2024-09-16T11:45:39.642890+0000","last_became_peered":"2024-09-16T11:45:39.642890+0000","last_unstale":"2024-09-16T11:46:07.379139+0000","last_undegraded":"2024-09-16T11:46:07.379139+0000","last_fullsized":"2024-09-16T11:46:07.379139+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'547","last_scrub_stamp":"2024-09-16T11:46:07.379005+0000","last_deep_scrub":"147'547","last_deep_scrub_stamp":"2024-09-16T11:46:07.379005+0000","last_clean_scrub_stamp":"2024-09-16T11:46:07.379005+0000","objects_scrubbed":22,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:01.499946+0000","scrub_duration":0.014758689,"objects_trimmed":2,"snaptrim_duration":0.0062520060000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4283,"num_read_kb":14493,"num_write":541,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"132'743","reported_seq":6418,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.709209+0000","last_change":"2024-09-16T11:45:39.656713+0000","last_active":"2024-09-16T11:45:54.709209+0000","last_peered":"2024-09-16T11:45:54.709209+0000","last_clean":"2024-09-16T11:45:54.709209+0000","last_became_active":"2024-09-16T11:45:39.640293+0000","last_became_peered":"2024-09-16T11:45:39.640293+0000","last_unstale":"2024-09-16T11:45:54.709209+0000","last_undegraded":"2024-09-16T11:45:54.709209+0000","last_fullsized":"2024-09-16T11:45:54.709209+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074727580000000004,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8298,"num_read_kb":47234,"num_write":833,"num_write_kb":15319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"147'687","reported_seq":2856,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:16.947658+0000","last_change":"2024-09-16T11:46:16.947626+0000","last_active":"2024-09-16T11:46:16.947658+0000","last_peered":"2024-09-16T11:46:16.947658+0000","last_clean":"2024-09-16T11:46:16.947658+0000","last_became_active":"2024-09-16T11:43:31.889629+0000","last_became_peered":"2024-09-16T11:43:31.889629+0000","last_unstale":"2024-09-16T11:46:16.947658+0000","last_undegraded":"2024-09-16T11:46:16.947658+0000","last_fullsized":"2024-09-16T11:46:16.947658+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'687","last_scrub_stamp":"2024-09-16T11:46:16.947549+0000","last_deep_scrub":"147'687","last_deep_scrub_stamp":"2024-09-16T11:46:16.947549+0000","last_clean_scrub_stamp":"2024-09-16T11:46:16.947549+0000","objects_scrubbed":30,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:26.385548+0000","scrub_duration":0.0220641,"objects_trimmed":0,"snaptrim_duration":0.00030057499999999997,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":3982,"num_read_kb":26490,"num_write":811,"num_write_kb":16305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"147'687","reported_seq":3494,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:20.369085+0000","last_change":"2024-09-16T11:46:20.369043+0000","last_active":"2024-09-16T11:46:20.369085+0000","last_peered":"2024-09-16T11:46:20.369085+0000","last_clean":"2024-09-16T11:46:20.369085+0000","last_became_active":"2024-09-16T11:45:39.641187+0000","last_became_peered":"2024-09-16T11:45:39.641187+0000","last_unstale":"2024-09-16T11:46:20.369085+0000","last_undegraded":"2024-09-16T11:46:20.369085+0000","last_fullsized":"2024-09-16T11:46:20.369085+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'687","last_scrub_stamp":"2024-09-16T11:46:20.368945+0000","last_deep_scrub":"147'687","last_deep_scrub_stamp":"2024-09-16T11:46:20.368945+0000","last_clean_scrub_stamp":"2024-09-16T11:46:20.368945+0000","objects_scrubbed":22,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:48.585782+0000","scrub_duration":0.016326818,"objects_trimmed":2,"snaptrim_duration":0.0074563379999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4363,"num_read_kb":20582,"num_write":914,"num_write_kb":10744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":3197941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"147'569","reported_seq":2545,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:08.145807+0000","last_change":"2024-09-16T11:46:08.145768+0000","last_active":"2024-09-16T11:46:08.145807+0000","last_peered":"2024-09-16T11:46:08.145807+0000","last_clean":"2024-09-16T11:46:08.145807+0000","last_became_active":"2024-09-16T11:43:31.776781+0000","last_became_peered":"2024-09-16T11:43:31.776781+0000","last_unstale":"2024-09-16T11:46:08.145807+0000","last_undegraded":"2024-09-16T11:46:08.145807+0000","last_fullsized":"2024-09-16T11:46:08.145807+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'569","last_scrub_stamp":"2024-09-16T11:46:08.145641+0000","last_deep_scrub":"147'569","last_deep_scrub_stamp":"2024-09-16T11:46:08.145641+0000","last_clean_scrub_stamp":"2024-09-16T11:46:08.145641+0000","objects_scrubbed":23,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:45.614029+0000","scrub_duration":0.01907329,"objects_trimmed":0,"snaptrim_duration":0.00032933499999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3911,"num_read_kb":15179,"num_write":595,"num_write_kb":10620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"121'294","reported_seq":1909,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.962563+0000","last_change":"2024-09-16T11:45:19.642323+0000","last_active":"2024-09-16T11:45:54.962563+0000","last_peered":"2024-09-16T11:45:54.962563+0000","last_clean":"2024-09-16T11:45:54.962563+0000","last_became_active":"2024-09-16T11:43:31.780164+0000","last_became_peered":"2024-09-16T11:43:31.780164+0000","last_unstale":"2024-09-16T11:45:54.962563+0000","last_undegraded":"2024-09-16T11:45:54.962563+0000","last_fullsized":"2024-09-16T11:45:54.962563+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079981,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1687,"num_read_kb":10324,"num_write":299,"num_write_kb":6058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"147'844","reported_seq":2312,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:09.382654+0000","last_change":"2024-09-16T11:46:09.382609+0000","last_active":"2024-09-16T11:46:09.382654+0000","last_peered":"2024-09-16T11:46:09.382654+0000","last_clean":"2024-09-16T11:46:09.382654+0000","last_became_active":"2024-09-16T11:45:39.641126+0000","last_became_peered":"2024-09-16T11:45:39.641126+0000","last_unstale":"2024-09-16T11:46:09.382654+0000","last_undegraded":"2024-09-16T11:46:09.382654+0000","last_fullsized":"2024-09-16T11:46:09.382654+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'844","last_scrub_stamp":"2024-09-16T11:46:09.382515+0000","last_deep_scrub":"147'844","last_deep_scrub_stamp":"2024-09-16T11:46:09.382515+0000","last_clean_scrub_stamp":"2024-09-16T11:46:09.382515+0000","objects_scrubbed":40,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:20.750189+0000","scrub_duration":0.029339613,"objects_trimmed":2,"snaptrim_duration":0.0074597470000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":4259,"num_read_kb":18254,"num_write":1055,"num_write_kb":13061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"147'426","reported_seq":1945,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:13.341933+0000","last_change":"2024-09-16T11:46:13.341894+0000","last_active":"2024-09-16T11:46:13.341933+0000","last_peered":"2024-09-16T11:46:13.341933+0000","last_clean":"2024-09-16T11:46:13.341933+0000","last_became_active":"2024-09-16T11:45:39.637293+0000","last_became_peered":"2024-09-16T11:45:39.637293+0000","last_unstale":"2024-09-16T11:46:13.341933+0000","last_undegraded":"2024-09-16T11:46:13.341933+0000","last_fullsized":"2024-09-16T11:46:13.341933+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'426","last_scrub_stamp":"2024-09-16T11:46:13.341770+0000","last_deep_scrub":"147'426","last_deep_scrub_stamp":"2024-09-16T11:46:13.341770+0000","last_clean_scrub_stamp":"2024-09-16T11:46:13.341770+0000","objects_scrubbed":25,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:59.826641+0000","scrub_duration":0.018777958000000001,"objects_trimmed":4,"snaptrim_duration":0.016205368000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3249,"num_read_kb":9637,"num_write":346,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":150,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:10.823125+0000","last_change":"2024-09-16T11:46:10.823082+0000","last_active":"2024-09-16T11:46:10.823125+0000","last_peered":"2024-09-16T11:46:10.823125+0000","last_clean":"2024-09-16T11:46:10.823125+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:46:10.823125+0000","last_undegraded":"2024-09-16T11:46:10.823125+0000","last_fullsized":"2024-09-16T11:46:10.823125+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:10.822991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:10.822991+0000","last_clean_scrub_stamp":"2024-09-16T11:46:10.822991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:54:42.610189+0000","scrub_duration":0.001279783,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"125'347","reported_seq":1762,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.725979+0000","last_change":"2024-09-16T11:45:47.465149+0000","last_active":"2024-09-16T11:45:54.725979+0000","last_peered":"2024-09-16T11:45:54.725979+0000","last_clean":"2024-09-16T11:45:54.725979+0000","last_became_active":"2024-09-16T11:45:47.464115+0000","last_became_peered":"2024-09-16T11:45:47.464115+0000","last_unstale":"2024-09-16T11:45:54.725979+0000","last_undegraded":"2024-09-16T11:45:54.725979+0000","last_fullsized":"2024-09-16T11:45:54.725979+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00038392500000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2491,"num_read_kb":9150,"num_write":283,"num_write_kb":9998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":152,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:10.560106+0000","last_change":"2024-09-16T11:46:10.560058+0000","last_active":"2024-09-16T11:46:10.560106+0000","last_peered":"2024-09-16T11:46:10.560106+0000","last_clean":"2024-09-16T11:46:10.560106+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:46:10.560106+0000","last_undegraded":"2024-09-16T11:46:10.560106+0000","last_fullsized":"2024-09-16T11:46:10.560106+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:10.559951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:10.559951+0000","last_clean_scrub_stamp":"2024-09-16T11:46:10.559951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:40.196781+0000","scrub_duration":0.0018533320000000001,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"147'500","reported_seq":4615,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:14.383623+0000","last_change":"2024-09-16T11:46:14.383596+0000","last_active":"2024-09-16T11:46:14.383623+0000","last_peered":"2024-09-16T11:46:14.383623+0000","last_clean":"2024-09-16T11:46:14.383623+0000","last_became_active":"2024-09-16T11:45:39.641298+0000","last_became_peered":"2024-09-16T11:45:39.641298+0000","last_unstale":"2024-09-16T11:46:14.383623+0000","last_undegraded":"2024-09-16T11:46:14.383623+0000","last_fullsized":"2024-09-16T11:46:14.383623+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'500","last_scrub_stamp":"2024-09-16T11:46:14.383501+0000","last_deep_scrub":"147'500","last_deep_scrub_stamp":"2024-09-16T11:46:14.383501+0000","last_clean_scrub_stamp":"2024-09-16T11:46:14.383501+0000","objects_scrubbed":23,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:51:05.634633+0000","scrub_duration":0.016287149000000001,"objects_trimmed":6,"snaptrim_duration":0.0094940819999999992,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5544,"num_read_kb":23694,"num_write":515,"num_write_kb":8223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":150,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:11.278804+0000","last_change":"2024-09-16T11:46:11.278740+0000","last_active":"2024-09-16T11:46:11.278804+0000","last_peered":"2024-09-16T11:46:11.278804+0000","last_clean":"2024-09-16T11:46:11.278804+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:46:11.278804+0000","last_undegraded":"2024-09-16T11:46:11.278804+0000","last_fullsized":"2024-09-16T11:46:11.278804+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:11.278624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:11.278624+0000","last_clean_scrub_stamp":"2024-09-16T11:46:11.278624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:01.706616+0000","scrub_duration":0.0020868729999999999,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"147'646","reported_seq":6897,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:15.574106+0000","last_change":"2024-09-16T11:46:15.574056+0000","last_active":"2024-09-16T11:46:15.574106+0000","last_peered":"2024-09-16T11:46:15.574106+0000","last_clean":"2024-09-16T11:46:15.574106+0000","last_became_active":"2024-09-16T11:43:31.777896+0000","last_became_peered":"2024-09-16T11:43:31.777896+0000","last_unstale":"2024-09-16T11:46:15.574106+0000","last_undegraded":"2024-09-16T11:46:15.574106+0000","last_fullsized":"2024-09-16T11:46:15.574106+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'646","last_scrub_stamp":"2024-09-16T11:46:15.573965+0000","last_deep_scrub":"147'646","last_deep_scrub_stamp":"2024-09-16T11:46:15.573965+0000","last_clean_scrub_stamp":"2024-09-16T11:46:15.573965+0000","objects_scrubbed":19,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:16.367155+0000","scrub_duration":0.014842227,"objects_trimmed":0,"snaptrim_duration":0.0011770719999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":7410,"num_read_kb":47228,"num_write":617,"num_write_kb":8894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":152,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:03.502189+0000","last_change":"2024-09-16T11:46:03.502167+0000","last_active":"2024-09-16T11:46:03.502189+0000","last_peered":"2024-09-16T11:46:03.502189+0000","last_clean":"2024-09-16T11:46:03.502189+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:46:03.502189+0000","last_undegraded":"2024-09-16T11:46:03.502189+0000","last_fullsized":"2024-09-16T11:46:03.502189+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:03.502083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:03.502083+0000","last_clean_scrub_stamp":"2024-09-16T11:46:03.502083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:51.618689+0000","scrub_duration":0.0011994970000000001,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"126'382","reported_seq":1084,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.962611+0000","last_change":"2024-09-16T11:45:19.642384+0000","last_active":"2024-09-16T11:45:54.962611+0000","last_peered":"2024-09-16T11:45:54.962611+0000","last_clean":"2024-09-16T11:45:54.962611+0000","last_became_active":"2024-09-16T11:43:31.778506+0000","last_became_peered":"2024-09-16T11:43:31.778506+0000","last_unstale":"2024-09-16T11:45:54.962611+0000","last_undegraded":"2024-09-16T11:45:54.962611+0000","last_fullsized":"2024-09-16T11:45:54.962611+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011478079999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1438,"num_read_kb":4478,"num_write":491,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":121,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:12.209849+0000","last_change":"2024-09-16T11:46:12.209821+0000","last_active":"2024-09-16T11:46:12.209849+0000","last_peered":"2024-09-16T11:46:12.209849+0000","last_clean":"2024-09-16T11:46:12.209849+0000","last_became_active":"2024-09-16T11:45:47.462295+0000","last_became_peered":"2024-09-16T11:45:47.462295+0000","last_unstale":"2024-09-16T11:46:12.209849+0000","last_undegraded":"2024-09-16T11:46:12.209849+0000","last_fullsized":"2024-09-16T11:46:12.209849+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:12.209728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:12.209728+0000","last_clean_scrub_stamp":"2024-09-16T11:46:12.209728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:49.726383+0000","scrub_duration":0.001514369,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"147'3","reported_seq":152,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:07.839398+0000","last_change":"2024-09-16T11:46:07.839348+0000","last_active":"2024-09-16T11:46:07.839398+0000","last_peered":"2024-09-16T11:46:07.839398+0000","last_clean":"2024-09-16T11:46:07.839398+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:46:07.839398+0000","last_undegraded":"2024-09-16T11:46:07.839398+0000","last_fullsized":"2024-09-16T11:46:07.839398+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'3","last_scrub_stamp":"2024-09-16T11:46:07.839246+0000","last_deep_scrub":"147'3","last_deep_scrub_stamp":"2024-09-16T11:46:07.839246+0000","last_clean_scrub_stamp":"2024-09-16T11:46:07.839246+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:53.638295+0000","scrub_duration":0.003326922,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"127'734","reported_seq":2604,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.962693+0000","last_change":"2024-09-16T11:45:19.641341+0000","last_active":"2024-09-16T11:45:54.962693+0000","last_peered":"2024-09-16T11:45:54.962693+0000","last_clean":"2024-09-16T11:45:54.962693+0000","last_became_active":"2024-09-16T11:43:31.777796+0000","last_became_peered":"2024-09-16T11:43:31.777796+0000","last_unstale":"2024-09-16T11:45:54.962693+0000","last_undegraded":"2024-09-16T11:45:54.962693+0000","last_fullsized":"2024-09-16T11:45:54.962693+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051098400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4214,"num_read_kb":15199,"num_write":913,"num_write_kb":10139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:19.703058+0000","last_change":"2024-09-16T11:46:19.703033+0000","last_active":"2024-09-16T11:46:19.703058+0000","last_peered":"2024-09-16T11:46:19.703058+0000","last_clean":"2024-09-16T11:46:19.703058+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:46:19.703058+0000","last_undegraded":"2024-09-16T11:46:19.703058+0000","last_fullsized":"2024-09-16T11:46:19.703058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:19.702943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:19.702943+0000","last_clean_scrub_stamp":"2024-09-16T11:46:19.702943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:12:02.946224+0000","scrub_duration":0.001388046,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":169,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:04.489635+0000","last_change":"2024-09-16T11:46:04.489617+0000","last_active":"2024-09-16T11:46:04.489635+0000","last_peered":"2024-09-16T11:46:04.489635+0000","last_clean":"2024-09-16T11:46:04.489635+0000","last_became_active":"2024-09-16T11:45:39.635334+0000","last_became_peered":"2024-09-16T11:45:39.635334+0000","last_unstale":"2024-09-16T11:46:04.489635+0000","last_undegraded":"2024-09-16T11:46:04.489635+0000","last_fullsized":"2024-09-16T11:46:04.489635+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:04.489545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:04.489545+0000","last_clean_scrub_stamp":"2024-09-16T11:46:04.489545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:38.788913+0000","scrub_duration":0.00112024,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"147'569","reported_seq":3424,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:22.617289+0000","last_change":"2024-09-16T11:46:22.617240+0000","last_active":"2024-09-16T11:46:22.617289+0000","last_peered":"2024-09-16T11:46:22.617289+0000","last_clean":"2024-09-16T11:46:22.617289+0000","last_became_active":"2024-09-16T11:43:31.777967+0000","last_became_peered":"2024-09-16T11:43:31.777967+0000","last_unstale":"2024-09-16T11:46:22.617289+0000","last_undegraded":"2024-09-16T11:46:22.617289+0000","last_fullsized":"2024-09-16T11:46:22.617289+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'569","last_scrub_stamp":"2024-09-16T11:46:22.617140+0000","last_deep_scrub":"147'569","last_deep_scrub_stamp":"2024-09-16T11:46:22.617140+0000","last_clean_scrub_stamp":"2024-09-16T11:46:22.617140+0000","objects_scrubbed":23,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:17:13.197926+0000","scrub_duration":0.016575501999999999,"objects_trimmed":0,"snaptrim_duration":0.00081483900000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3645,"num_read_kb":23388,"num_write":630,"num_write_kb":9618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"147'5","reported_seq":147,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:11.474783+0000","last_change":"2024-09-16T11:46:11.474728+0000","last_active":"2024-09-16T11:46:11.474783+0000","last_peered":"2024-09-16T11:46:11.474783+0000","last_clean":"2024-09-16T11:46:11.474783+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:46:11.474783+0000","last_undegraded":"2024-09-16T11:46:11.474783+0000","last_fullsized":"2024-09-16T11:46:11.474783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'4","last_scrub_stamp":"2024-09-16T11:46:11.474608+0000","last_deep_scrub":"147'4","last_deep_scrub_stamp":"2024-09-16T11:46:11.474608+0000","last_clean_scrub_stamp":"2024-09-16T11:46:11.474608+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:09:01.168170+0000","scrub_duration":0.0038928880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":166,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":150,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:07.323759+0000","last_change":"2024-09-16T11:46:07.323734+0000","last_active":"2024-09-16T11:46:07.323759+0000","last_peered":"2024-09-16T11:46:07.323759+0000","last_clean":"2024-09-16T11:46:07.323759+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:46:07.323759+0000","last_undegraded":"2024-09-16T11:46:07.323759+0000","last_fullsized":"2024-09-16T11:46:07.323759+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:07.323641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:07.323641+0000","last_clean_scrub_stamp":"2024-09-16T11:46:07.323641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:09.437604+0000","scrub_duration":0.0020561540000000001,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"147'427","reported_seq":1801,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:17.591254+0000","last_change":"2024-09-16T11:46:17.591209+0000","last_active":"2024-09-16T11:46:17.591254+0000","last_peered":"2024-09-16T11:46:17.591254+0000","last_clean":"2024-09-16T11:46:17.591254+0000","last_became_active":"2024-09-16T11:43:31.776645+0000","last_became_peered":"2024-09-16T11:43:31.776645+0000","last_unstale":"2024-09-16T11:46:17.591254+0000","last_undegraded":"2024-09-16T11:46:17.591254+0000","last_fullsized":"2024-09-16T11:46:17.591254+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'427","last_scrub_stamp":"2024-09-16T11:46:17.591116+0000","last_deep_scrub":"147'427","last_deep_scrub_stamp":"2024-09-16T11:46:17.591116+0000","last_clean_scrub_stamp":"2024-09-16T11:46:17.591116+0000","objects_scrubbed":20,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:55.287547+0000","scrub_duration":0.013905288,"objects_trimmed":0,"snaptrim_duration":0.00066659399999999995,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1650,"num_read_kb":12114,"num_write":331,"num_write_kb":15245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"147'6","reported_seq":111,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:06.411314+0000","last_change":"2024-09-16T11:46:06.411284+0000","last_active":"2024-09-16T11:46:06.411314+0000","last_peered":"2024-09-16T11:46:06.411314+0000","last_clean":"2024-09-16T11:46:06.411314+0000","last_became_active":"2024-09-16T11:45:39.637629+0000","last_became_peered":"2024-09-16T11:45:39.637629+0000","last_unstale":"2024-09-16T11:46:06.411314+0000","last_undegraded":"2024-09-16T11:46:06.411314+0000","last_fullsized":"2024-09-16T11:46:06.411314+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'6","last_scrub_stamp":"2024-09-16T11:46:06.411204+0000","last_deep_scrub":"147'6","last_deep_scrub_stamp":"2024-09-16T11:46:06.411204+0000","last_clean_scrub_stamp":"2024-09-16T11:46:06.411204+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:03.945238+0000","scrub_duration":0.0038620859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"147'2","reported_seq":151,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:10.884409+0000","last_change":"2024-09-16T11:46:10.884380+0000","last_active":"2024-09-16T11:46:10.884409+0000","last_peered":"2024-09-16T11:46:10.884409+0000","last_clean":"2024-09-16T11:46:10.884409+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:46:10.884409+0000","last_undegraded":"2024-09-16T11:46:10.884409+0000","last_fullsized":"2024-09-16T11:46:10.884409+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'2","last_scrub_stamp":"2024-09-16T11:46:10.884282+0000","last_deep_scrub":"147'2","last_deep_scrub_stamp":"2024-09-16T11:46:10.884282+0000","last_clean_scrub_stamp":"2024-09-16T11:46:10.884282+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:38.365702+0000","scrub_duration":0.0031331610000000002,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"147'74","reported_seq":239,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:06.277326+0000","last_change":"2024-09-16T11:46:06.277299+0000","last_active":"2024-09-16T11:46:06.277326+0000","last_peered":"2024-09-16T11:46:06.277326+0000","last_clean":"2024-09-16T11:46:06.277326+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:46:06.277326+0000","last_undegraded":"2024-09-16T11:46:06.277326+0000","last_fullsized":"2024-09-16T11:46:06.277326+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'74","last_scrub_stamp":"2024-09-16T11:46:06.277205+0000","last_deep_scrub":"147'74","last_deep_scrub_stamp":"2024-09-16T11:46:06.277205+0000","last_clean_scrub_stamp":"2024-09-16T11:46:06.277205+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:54.267542+0000","scrub_duration":0.0041307949999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"147'544","reported_seq":3470,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:11.844641+0000","last_change":"2024-09-16T11:46:11.844611+0000","last_active":"2024-09-16T11:46:11.844641+0000","last_peered":"2024-09-16T11:46:11.844641+0000","last_clean":"2024-09-16T11:46:11.844641+0000","last_became_active":"2024-09-16T11:43:31.782071+0000","last_became_peered":"2024-09-16T11:43:31.782071+0000","last_unstale":"2024-09-16T11:46:11.844641+0000","last_undegraded":"2024-09-16T11:46:11.844641+0000","last_fullsized":"2024-09-16T11:46:11.844641+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'544","last_scrub_stamp":"2024-09-16T11:46:11.844514+0000","last_deep_scrub":"147'544","last_deep_scrub_stamp":"2024-09-16T11:46:11.844514+0000","last_clean_scrub_stamp":"2024-09-16T11:46:11.844514+0000","objects_scrubbed":21,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:10.626670+0000","scrub_duration":0.01463768,"objects_trimmed":0,"snaptrim_duration":0.000599604,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3801,"num_read_kb":24361,"num_write":540,"num_write_kb":7978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"127'665","reported_seq":9007,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.962468+0000","last_change":"2024-09-16T11:45:19.641414+0000","last_active":"2024-09-16T11:45:54.962468+0000","last_peered":"2024-09-16T11:45:54.962468+0000","last_clean":"2024-09-16T11:45:54.962468+0000","last_became_active":"2024-09-16T11:44:10.920131+0000","last_became_peered":"2024-09-16T11:44:10.920131+0000","last_unstale":"2024-09-16T11:45:54.962468+0000","last_undegraded":"2024-09-16T11:45:54.962468+0000","last_fullsized":"2024-09-16T11:45:54.962468+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082161099999999998,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":9498,"num_read_kb":69976,"num_write":712,"num_write_kb":19149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":8960357,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"147'904","reported_seq":3899,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:19.178394+0000","last_change":"2024-09-16T11:46:19.178350+0000","last_active":"2024-09-16T11:46:19.178394+0000","last_peered":"2024-09-16T11:46:19.178394+0000","last_clean":"2024-09-16T11:46:19.178394+0000","last_became_active":"2024-09-16T11:45:47.464353+0000","last_became_peered":"2024-09-16T11:45:47.464353+0000","last_unstale":"2024-09-16T11:46:19.178394+0000","last_undegraded":"2024-09-16T11:46:19.178394+0000","last_fullsized":"2024-09-16T11:46:19.178394+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'904","last_scrub_stamp":"2024-09-16T11:46:19.178262+0000","last_deep_scrub":"147'904","last_deep_scrub_stamp":"2024-09-16T11:46:19.178262+0000","last_clean_scrub_stamp":"2024-09-16T11:46:19.178262+0000","objects_scrubbed":26,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:10:26.656285+0000","scrub_duration":0.020921347,"objects_trimmed":0,"snaptrim_duration":0.001068741,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":8005,"num_read_kb":12353,"num_write":894,"num_write_kb":6127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"147'655","reported_seq":1861,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:15.836157+0000","last_change":"2024-09-16T11:46:15.836120+0000","last_active":"2024-09-16T11:46:15.836157+0000","last_peered":"2024-09-16T11:46:15.836157+0000","last_clean":"2024-09-16T11:46:15.836157+0000","last_became_active":"2024-09-16T11:43:31.779384+0000","last_became_peered":"2024-09-16T11:43:31.779384+0000","last_unstale":"2024-09-16T11:46:15.836157+0000","last_undegraded":"2024-09-16T11:46:15.836157+0000","last_fullsized":"2024-09-16T11:46:15.836157+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'655","last_scrub_stamp":"2024-09-16T11:46:15.836039+0000","last_deep_scrub":"147'655","last_deep_scrub_stamp":"2024-09-16T11:46:15.836039+0000","last_clean_scrub_stamp":"2024-09-16T11:46:15.836039+0000","objects_scrubbed":32,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:22:30.289390+0000","scrub_duration":0.021988002999999999,"objects_trimmed":0,"snaptrim_duration":0.000647592,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":2523,"num_read_kb":9199,"num_write":666,"num_write_kb":6203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"125'377","reported_seq":3381,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.708999+0000","last_change":"2024-09-16T11:45:19.593453+0000","last_active":"2024-09-16T11:45:54.708999+0000","last_peered":"2024-09-16T11:45:54.708999+0000","last_clean":"2024-09-16T11:45:54.708999+0000","last_became_active":"2024-09-16T11:43:31.775335+0000","last_became_peered":"2024-09-16T11:43:31.775335+0000","last_unstale":"2024-09-16T11:45:54.708999+0000","last_undegraded":"2024-09-16T11:45:54.708999+0000","last_fullsized":"2024-09-16T11:45:54.708999+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011352389999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4122,"num_read_kb":15191,"num_write":417,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"147'545","reported_seq":2303,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:20.171393+0000","last_change":"2024-09-16T11:46:20.171358+0000","last_active":"2024-09-16T11:46:20.171393+0000","last_peered":"2024-09-16T11:46:20.171393+0000","last_clean":"2024-09-16T11:46:20.171393+0000","last_became_active":"2024-09-16T11:45:47.463487+0000","last_became_peered":"2024-09-16T11:45:47.463487+0000","last_unstale":"2024-09-16T11:46:20.171393+0000","last_undegraded":"2024-09-16T11:46:20.171393+0000","last_fullsized":"2024-09-16T11:46:20.171393+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'545","last_scrub_stamp":"2024-09-16T11:46:20.171264+0000","last_deep_scrub":"147'545","last_deep_scrub_stamp":"2024-09-16T11:46:20.171264+0000","last_clean_scrub_stamp":"2024-09-16T11:46:20.171264+0000","objects_scrubbed":24,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:05:39.743556+0000","scrub_duration":0.015237156999999999,"objects_trimmed":2,"snaptrim_duration":0.001754511,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4037,"num_read_kb":5871,"num_write":454,"num_write_kb":2926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"125'616","reported_seq":5074,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.726134+0000","last_change":"2024-09-16T11:45:47.462731+0000","last_active":"2024-09-16T11:45:54.726134+0000","last_peered":"2024-09-16T11:45:54.726134+0000","last_clean":"2024-09-16T11:45:54.726134+0000","last_became_active":"2024-09-16T11:45:47.461210+0000","last_became_peered":"2024-09-16T11:45:47.461210+0000","last_unstale":"2024-09-16T11:45:54.726134+0000","last_undegraded":"2024-09-16T11:45:54.726134+0000","last_fullsized":"2024-09-16T11:45:54.726134+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00043040400000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6463,"num_read_kb":36386,"num_write":811,"num_write_kb":14056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"147'628","reported_seq":1965,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:11.204020+0000","last_change":"2024-09-16T11:46:11.203985+0000","last_active":"2024-09-16T11:46:11.204020+0000","last_peered":"2024-09-16T11:46:11.204020+0000","last_clean":"2024-09-16T11:46:11.204020+0000","last_became_active":"2024-09-16T11:45:47.464052+0000","last_became_peered":"2024-09-16T11:45:47.464052+0000","last_unstale":"2024-09-16T11:46:11.204020+0000","last_undegraded":"2024-09-16T11:46:11.204020+0000","last_fullsized":"2024-09-16T11:46:11.204020+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'628","last_scrub_stamp":"2024-09-16T11:46:11.203896+0000","last_deep_scrub":"147'628","last_deep_scrub_stamp":"2024-09-16T11:46:11.203896+0000","last_clean_scrub_stamp":"2024-09-16T11:46:11.203896+0000","objects_scrubbed":23,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:36.692499+0000","scrub_duration":0.016278948000000001,"objects_trimmed":0,"snaptrim_duration":0.00059308799999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2764,"num_read_kb":9887,"num_write":657,"num_write_kb":5333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"127'408","reported_seq":2118,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:45:54.962748+0000","last_change":"2024-09-16T11:45:39.667068+0000","last_active":"2024-09-16T11:45:54.962748+0000","last_peered":"2024-09-16T11:45:54.962748+0000","last_clean":"2024-09-16T11:45:54.962748+0000","last_became_active":"2024-09-16T11:45:39.643207+0000","last_became_peered":"2024-09-16T11:45:39.643207+0000","last_unstale":"2024-09-16T11:45:54.962748+0000","last_undegraded":"2024-09-16T11:45:54.962748+0000","last_fullsized":"2024-09-16T11:45:54.962748+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:43:30.758179+0000","last_clean_scrub_stamp":"2024-09-16T11:43:30.758179+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070987940000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3609,"num_read_kb":5161,"num_write":362,"num_write_kb":8941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":728,"num_read":135475,"num_read_kb":642896,"num_write":18060,"num_write_kb":290606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},"log_size":16939,"ondisk_log_size":16939,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":251,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":251,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299124,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072716,"statfs":{"total":96636764160,"available":96330461184,"internally_reserved":0,"allocated":1814528,"data_stored":1481880,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.46500000000000002,"15min":0.45100000000000001},"min":{"1min":0.26400000000000001,"5min":0.182,"15min":0.182},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.497,"15min":0.48199999999999998},"min":{"1min":0.33100000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59199999999999997}]},{"osd":1,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.61799999999999999},"min":{"1min":0.42299999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.91900000000000004,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.46400000000000002,"15min":0.44400000000000001},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.57599999999999996}]},{"osd":2,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.38,"5min":0.248,"15min":0.248},"max":{"1min":0.68500000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.505},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.437,"5min":0.309,"15min":0.309},"max":{"1min":0.86399999999999999,"5min":1.532,"15min":1.532},"last":0.55900000000000005}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.5640000000000001,"5min":3.5640000000000001,"15min":3.5640000000000001},"last":0.48999999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.435,"15min":0.44500000000000001},"min":{"1min":0.32900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.65100000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.378},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.45700000000000002},"min":{"1min":0.28000000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.57299999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40699999999999997}]},{"osd":5,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.432,"15min":0.438},"min":{"1min":0.32000000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48899999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.47299999999999998,"15min":0.48299999999999998},"min":{"1min":0.33600000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.59199999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.42799999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.48299999999999998,"15min":0.47399999999999998},"min":{"1min":0.38700000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47099999999999997,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61699999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.46300000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072760,"statfs":{"total":96636764160,"available":96330506240,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.33300000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66900000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.315,"5min":0.252,"15min":0.252},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.59299999999999997}]},{"osd":1,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45200000000000001,"15min":0.44800000000000001},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.39000000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65800000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.65100000000000002}]},{"osd":2,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.46800000000000003,"15min":0.46100000000000002},"min":{"1min":0.38200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.36799999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69499999999999995,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.66400000000000003}]},{"osd":3,"last update":"Mon Sep 16 11:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.625}]},{"osd":4,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.309,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63600000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.45500000000000002},"min":{"1min":0.34499999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.42699999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.45100000000000001,"15min":0.45900000000000002},"min":{"1min":0.29099999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64600000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.36799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.501}]},{"osd":7,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.34499999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.501,"15min":0.51100000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.64800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.60699999999999998}]}]},{"osd":3,"up_from":131,"seq":562640715787,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302152,"kb_used_data":1728,"kb_used_omap":43,"kb_used_meta":300372,"kb_avail":94069688,"statfs":{"total":96636764160,"available":96327360512,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44228,"internal_metadata":307581756},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299076,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072764,"statfs":{"total":96636764160,"available":96330510336,"internally_reserved":0,"allocated":1765376,"data_stored":892439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.33200000000000002,"15min":0.32700000000000001},"min":{"1min":0.20699999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.48999999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.495,"15min":0.39700000000000002},"min":{"1min":0.219,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.60099999999999998,"5min":11.218,"15min":11.218},"last":0.33400000000000002}]},{"osd":2,"last update":"Mon Sep 16 11:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.32400000000000001},"min":{"1min":0.26000000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.50700000000000001,"5min":0.746,"15min":0.746},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.45200000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.41199999999999998}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.376},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.34100000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.318,"5min":0.315,"15min":0.315},"max":{"1min":0.81100000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85899999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.41999999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76900000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.54000000000000004,"15min":0.53000000000000003},"min":{"1min":0.441,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55400000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.436,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91700000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77900000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311548,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94060292,"statfs":{"total":96636764160,"available":96317739008,"internally_reserved":0,"allocated":1757184,"data_stored":891472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.45500000000000002,"15min":0.45600000000000002},"min":{"1min":0.379,"5min":0.223,"15min":0.223},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.439,"15min":0.441},"min":{"1min":0.35499999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.64300000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73699999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.41299999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.63200000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.42999999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63700000000000001,"5min":1.026,"15min":1.026},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.38900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.80000000000000004,"5min":1.01,"15min":1.01},"last":0.55200000000000005}]},{"osd":3,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.503}]},{"osd":4,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.40400000000000003},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54900000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.46500000000000002,"15min":0.46400000000000002},"min":{"1min":0.39000000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70699999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44800000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44800000000000001,"15min":0.46400000000000002},"min":{"1min":0.315,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60999999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.435},"min":{"1min":0.35099999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.46800000000000003}]},{"osd":7,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.33200000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.746,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48499999999999999,"15min":0.496},"min":{"1min":0.34799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.77300000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303244,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068596,"statfs":{"total":96636764160,"available":96326242304,"internally_reserved":0,"allocated":1773568,"data_stored":892605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48799999999999999,"15min":0.44500000000000001},"min":{"1min":0.31900000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.64500000000000002,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.434},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.46000000000000002,"15min":0.45500000000000002},"min":{"1min":0.374,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.60699999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.5}]},{"osd":1,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.47599999999999998,"15min":0.47099999999999997},"min":{"1min":0.32600000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.36299999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.52800000000000002}]},{"osd":2,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.49299999999999999,"15min":0.498},"min":{"1min":0.39400000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.501,"15min":0.503},"min":{"1min":0.38400000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Sep 16 11:44:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.68000000000000005}]},{"osd":5,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41699999999999998,"15min":0.433},"min":{"1min":0.27400000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.628,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.45200000000000001,"15min":0.46800000000000003},"min":{"1min":0.32800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.76400000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.624}]},{"osd":6,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.47899999999999998},"min":{"1min":0.29999999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66600000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.55000000000000004},"min":{"1min":0.38600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70299999999999996}]},{"osd":7,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.50600000000000001},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.79900000000000004,"5min":0.873,"15min":0.873},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.52300000000000002},"min":{"1min":0.34200000000000003,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77900000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299072,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072768,"statfs":{"total":96636764160,"available":96330514432,"internally_reserved":0,"allocated":1761280,"data_stored":892420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.308,"15min":0.30399999999999999},"min":{"1min":0.20599999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.377,"5min":0.33900000000000002,"15min":0.32900000000000001},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.51400000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.35699999999999998,"15min":0.35199999999999998},"min":{"1min":0.23599999999999999,"5min":0.192,"15min":0.192},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.42799999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:44:11 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.499}]},{"osd":4,"last update":"Mon Sep 16 11:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.58199999999999996,"15min":0.55500000000000005},"min":{"1min":0.28899999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.83899999999999997}]},{"osd":5,"last update":"Mon Sep 16 11:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33500000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2889999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.85499999999999998}]},{"osd":6,"last update":"Mon Sep 16 11:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.88600000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.39900000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.2689999999999999,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.58899999999999997,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.755}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299136,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072704,"statfs":{"total":96636764160,"available":96330448896,"internally_reserved":0,"allocated":1826816,"data_stored":1482954,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.39800000000000002,"15min":0.33800000000000002},"min":{"1min":0.19500000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.503,"5min":8.5050000000000008,"15min":8.5050000000000008},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.32900000000000001,"15min":0.32300000000000001},"min":{"1min":0.23799999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.48599999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.48599999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.40000000000000002,"15min":0.33700000000000002},"min":{"1min":0.25,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.50700000000000001,"5min":8.4689999999999994,"15min":8.4689999999999994},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.32600000000000001,"15min":0.32700000000000001},"min":{"1min":0.22900000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.44600000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.44800000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.57299999999999995}]},{"osd":4,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.52200000000000002},"min":{"1min":0.45400000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.746,"5min":8.5299999999999994,"15min":8.5299999999999994},"last":0.629},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52200000000000002,"15min":0.51200000000000001},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75700000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.78400000000000003}]},{"osd":5,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66400000000000003,"15min":0.60799999999999998},"min":{"1min":0.41399999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.125,"5min":8.6039999999999992,"15min":8.6039999999999992},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.38100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68700000000000006}]},{"osd":6,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57299999999999995,"15min":0.57399999999999995},"min":{"1min":0.42999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69099999999999995,"15min":0.63300000000000001},"min":{"1min":0.41099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.157,"5min":8.5809999999999995,"15min":8.5809999999999995},"last":0.84199999999999997}]},{"osd":7,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.39900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.753},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.625,"15min":0.55700000000000005},"min":{"1min":0.42399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73499999999999999,"5min":8.5570000000000004,"15min":8.5570000000000004},"last":0.86399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:46:26.208 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:26.208 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:26.208 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:26.209 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:26.209 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:26.209 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:26.209 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:26.209 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:26.209 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:26.210 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:26.210 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:26.210 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:26.210 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-16T11:43:30.758179+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=43, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=46, tm_sec=1, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:46:26.210 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T11:46:46.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:46:46.488 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T11:46:46.489 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T11:46:46.500 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-16T11:46:45.413177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590657,"num_objects":742,"num_object_clones":0,"num_object_copies":1484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":731,"num_read":135541,"num_read_kb":642951,"num_write":18166,"num_write_kb":292239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":251,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17309,"ondisk_log_size":17309,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2412432,"kb_used_data":13904,"kb_used_omap":54,"kb_used_meta":2398409,"kb_avail":752562288,"statfs":{"total":773094113280,"available":770623782912,"internally_reserved":0,"allocated":14237696,"data_stored":8318818,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56156,"internal_metadata":2455970980},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003047"},"pg_stats":[{"pgid":"4.1f","version":"147'340","reported_seq":6261,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:17.850091+0000","last_change":"2024-09-16T11:46:17.850036+0000","last_active":"2024-09-16T11:46:17.850091+0000","last_peered":"2024-09-16T11:46:17.850091+0000","last_clean":"2024-09-16T11:46:17.850091+0000","last_became_active":"2024-09-16T11:43:31.779428+0000","last_became_peered":"2024-09-16T11:43:31.779428+0000","last_unstale":"2024-09-16T11:46:17.850091+0000","last_undegraded":"2024-09-16T11:46:17.850091+0000","last_fullsized":"2024-09-16T11:46:17.850091+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'340","last_scrub_stamp":"2024-09-16T11:46:17.849933+0000","last_deep_scrub":"147'340","last_deep_scrub_stamp":"2024-09-16T11:46:17.849933+0000","last_clean_scrub_stamp":"2024-09-16T11:46:17.849933+0000","objects_scrubbed":13,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:04.745251+0000","scrub_duration":0.0093597230000000003,"objects_trimmed":0,"snaptrim_duration":0.00034041099999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6762,"num_read_kb":29123,"num_write":304,"num_write_kb":12360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"147'467","reported_seq":5375,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:27.410183+0000","last_change":"2024-09-16T11:46:27.410128+0000","last_active":"2024-09-16T11:46:27.410183+0000","last_peered":"2024-09-16T11:46:27.410183+0000","last_clean":"2024-09-16T11:46:27.410183+0000","last_became_active":"2024-09-16T11:43:31.889706+0000","last_became_peered":"2024-09-16T11:43:31.889706+0000","last_unstale":"2024-09-16T11:46:27.410183+0000","last_undegraded":"2024-09-16T11:46:27.410183+0000","last_fullsized":"2024-09-16T11:46:27.410183+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'467","last_scrub_stamp":"2024-09-16T11:46:27.410030+0000","last_deep_scrub":"147'467","last_deep_scrub_stamp":"2024-09-16T11:46:27.410030+0000","last_clean_scrub_stamp":"2024-09-16T11:46:27.410030+0000","objects_scrubbed":18,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:48:20.331551+0000","scrub_duration":0.016604710000000002,"objects_trimmed":0,"snaptrim_duration":0.000654113,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5672,"num_read_kb":36055,"num_write":486,"num_write_kb":9211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"147'233","reported_seq":1570,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:25.338367+0000","last_change":"2024-09-16T11:46:25.338319+0000","last_active":"2024-09-16T11:46:25.338367+0000","last_peered":"2024-09-16T11:46:25.338367+0000","last_clean":"2024-09-16T11:46:25.338367+0000","last_became_active":"2024-09-16T11:45:39.636105+0000","last_became_peered":"2024-09-16T11:45:39.636105+0000","last_unstale":"2024-09-16T11:46:25.338367+0000","last_undegraded":"2024-09-16T11:46:25.338367+0000","last_fullsized":"2024-09-16T11:46:25.338367+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'233","last_scrub_stamp":"2024-09-16T11:46:25.338223+0000","last_deep_scrub":"147'233","last_deep_scrub_stamp":"2024-09-16T11:46:25.338223+0000","last_clean_scrub_stamp":"2024-09-16T11:46:25.338223+0000","objects_scrubbed":20,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:47.164758+0000","scrub_duration":0.013074735000000001,"objects_trimmed":6,"snaptrim_duration":0.014051187999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1102,"num_read_kb":9776,"num_write":200,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"147'349","reported_seq":2439,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:05.481663+0000","last_change":"2024-09-16T11:46:05.481642+0000","last_active":"2024-09-16T11:46:05.481663+0000","last_peered":"2024-09-16T11:46:05.481663+0000","last_clean":"2024-09-16T11:46:05.481663+0000","last_became_active":"2024-09-16T11:43:31.773661+0000","last_became_peered":"2024-09-16T11:43:31.773661+0000","last_unstale":"2024-09-16T11:46:05.481663+0000","last_undegraded":"2024-09-16T11:46:05.481663+0000","last_fullsized":"2024-09-16T11:46:05.481663+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'349","last_scrub_stamp":"2024-09-16T11:46:05.481572+0000","last_deep_scrub":"147'349","last_deep_scrub_stamp":"2024-09-16T11:46:05.481572+0000","last_clean_scrub_stamp":"2024-09-16T11:46:05.481572+0000","objects_scrubbed":15,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:28:35.138063+0000","scrub_duration":0.0088023839999999999,"objects_trimmed":0,"snaptrim_duration":0.00052992100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":2191,"num_read_kb":14785,"num_write":375,"num_write_kb":7446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"147'407","reported_seq":2563,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:24.291124+0000","last_change":"2024-09-16T11:46:24.291083+0000","last_active":"2024-09-16T11:46:24.291124+0000","last_peered":"2024-09-16T11:46:24.291124+0000","last_clean":"2024-09-16T11:46:24.291124+0000","last_became_active":"2024-09-16T11:45:47.463209+0000","last_became_peered":"2024-09-16T11:45:47.463209+0000","last_unstale":"2024-09-16T11:46:24.291124+0000","last_undegraded":"2024-09-16T11:46:24.291124+0000","last_fullsized":"2024-09-16T11:46:24.291124+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'407","last_scrub_stamp":"2024-09-16T11:46:24.291000+0000","last_deep_scrub":"147'407","last_deep_scrub_stamp":"2024-09-16T11:46:24.291000+0000","last_clean_scrub_stamp":"2024-09-16T11:46:24.291000+0000","objects_scrubbed":27,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:30:03.625909+0000","scrub_duration":0.021265598,"objects_trimmed":6,"snaptrim_duration":0.0033897049999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3246,"num_read_kb":18406,"num_write":318,"num_write_kb":4718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"147'566","reported_seq":3561,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:18.137237+0000","last_change":"2024-09-16T11:46:18.137185+0000","last_active":"2024-09-16T11:46:18.137237+0000","last_peered":"2024-09-16T11:46:18.137237+0000","last_clean":"2024-09-16T11:46:18.137237+0000","last_became_active":"2024-09-16T11:45:47.463199+0000","last_became_peered":"2024-09-16T11:45:47.463199+0000","last_unstale":"2024-09-16T11:46:18.137237+0000","last_undegraded":"2024-09-16T11:46:18.137237+0000","last_fullsized":"2024-09-16T11:46:18.137237+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'566","last_scrub_stamp":"2024-09-16T11:46:18.137055+0000","last_deep_scrub":"147'566","last_deep_scrub_stamp":"2024-09-16T11:46:18.137055+0000","last_clean_scrub_stamp":"2024-09-16T11:46:18.137055+0000","objects_scrubbed":32,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:44.286191+0000","scrub_duration":0.027672068000000001,"objects_trimmed":12,"snaptrim_duration":0.01209723,"stat_sum":{"num_bytes":19,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":4102,"num_read_kb":12532,"num_write":637,"num_write_kb":9975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"147'675","reported_seq":4635,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:29.283010+0000","last_change":"2024-09-16T11:46:29.282981+0000","last_active":"2024-09-16T11:46:29.283010+0000","last_peered":"2024-09-16T11:46:29.283010+0000","last_clean":"2024-09-16T11:46:29.283010+0000","last_became_active":"2024-09-16T11:45:39.638104+0000","last_became_peered":"2024-09-16T11:45:39.638104+0000","last_unstale":"2024-09-16T11:46:29.283010+0000","last_undegraded":"2024-09-16T11:46:29.283010+0000","last_fullsized":"2024-09-16T11:46:29.283010+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'675","last_scrub_stamp":"2024-09-16T11:46:29.282892+0000","last_deep_scrub":"147'675","last_deep_scrub_stamp":"2024-09-16T11:46:29.282892+0000","last_clean_scrub_stamp":"2024-09-16T11:46:29.282892+0000","objects_scrubbed":17,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:10:31.304732+0000","scrub_duration":0.012240651999999999,"objects_trimmed":0,"snaptrim_duration":0.0069266390000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5632,"num_read_kb":29340,"num_write":868,"num_write_kb":4351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"147'240","reported_seq":1570,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:09.137966+0000","last_change":"2024-09-16T11:46:09.137936+0000","last_active":"2024-09-16T11:46:09.137966+0000","last_peered":"2024-09-16T11:46:09.137966+0000","last_clean":"2024-09-16T11:46:09.137966+0000","last_became_active":"2024-09-16T11:43:31.776951+0000","last_became_peered":"2024-09-16T11:43:31.776951+0000","last_unstale":"2024-09-16T11:46:09.137966+0000","last_undegraded":"2024-09-16T11:46:09.137966+0000","last_fullsized":"2024-09-16T11:46:09.137966+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'240","last_scrub_stamp":"2024-09-16T11:46:09.137849+0000","last_deep_scrub":"147'240","last_deep_scrub_stamp":"2024-09-16T11:46:09.137849+0000","last_clean_scrub_stamp":"2024-09-16T11:46:09.137849+0000","objects_scrubbed":19,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:00.683567+0000","scrub_duration":0.012833168000000001,"objects_trimmed":0,"snaptrim_duration":0.00052508100000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":1522,"num_read_kb":7050,"num_write":185,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"147'547","reported_seq":2374,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:07.379139+0000","last_change":"2024-09-16T11:46:07.379106+0000","last_active":"2024-09-16T11:46:07.379139+0000","last_peered":"2024-09-16T11:46:07.379139+0000","last_clean":"2024-09-16T11:46:07.379139+0000","last_became_active":"2024-09-16T11:45:39.642890+0000","last_became_peered":"2024-09-16T11:45:39.642890+0000","last_unstale":"2024-09-16T11:46:07.379139+0000","last_undegraded":"2024-09-16T11:46:07.379139+0000","last_fullsized":"2024-09-16T11:46:07.379139+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'547","last_scrub_stamp":"2024-09-16T11:46:07.379005+0000","last_deep_scrub":"147'547","last_deep_scrub_stamp":"2024-09-16T11:46:07.379005+0000","last_clean_scrub_stamp":"2024-09-16T11:46:07.379005+0000","objects_scrubbed":22,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:01.499946+0000","scrub_duration":0.014758689,"objects_trimmed":2,"snaptrim_duration":0.0062520060000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4283,"num_read_kb":14493,"num_write":541,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"147'770","reported_seq":6456,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:26.415372+0000","last_change":"2024-09-16T11:46:26.415347+0000","last_active":"2024-09-16T11:46:26.415372+0000","last_peered":"2024-09-16T11:46:26.415372+0000","last_clean":"2024-09-16T11:46:26.415372+0000","last_became_active":"2024-09-16T11:45:39.640293+0000","last_became_peered":"2024-09-16T11:45:39.640293+0000","last_unstale":"2024-09-16T11:46:26.415372+0000","last_undegraded":"2024-09-16T11:46:26.415372+0000","last_fullsized":"2024-09-16T11:46:26.415372+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'770","last_scrub_stamp":"2024-09-16T11:46:26.415271+0000","last_deep_scrub":"147'770","last_deep_scrub_stamp":"2024-09-16T11:46:26.415271+0000","last_clean_scrub_stamp":"2024-09-16T11:46:26.415271+0000","objects_scrubbed":27,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:02.138420+0000","scrub_duration":0.017912332,"objects_trimmed":2,"snaptrim_duration":0.0074727580000000004,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":8298,"num_read_kb":47234,"num_write":833,"num_write_kb":15319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"147'687","reported_seq":2856,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:16.947658+0000","last_change":"2024-09-16T11:46:16.947626+0000","last_active":"2024-09-16T11:46:16.947658+0000","last_peered":"2024-09-16T11:46:16.947658+0000","last_clean":"2024-09-16T11:46:16.947658+0000","last_became_active":"2024-09-16T11:43:31.889629+0000","last_became_peered":"2024-09-16T11:43:31.889629+0000","last_unstale":"2024-09-16T11:46:16.947658+0000","last_undegraded":"2024-09-16T11:46:16.947658+0000","last_fullsized":"2024-09-16T11:46:16.947658+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'687","last_scrub_stamp":"2024-09-16T11:46:16.947549+0000","last_deep_scrub":"147'687","last_deep_scrub_stamp":"2024-09-16T11:46:16.947549+0000","last_clean_scrub_stamp":"2024-09-16T11:46:16.947549+0000","objects_scrubbed":30,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:26.385548+0000","scrub_duration":0.0220641,"objects_trimmed":0,"snaptrim_duration":0.00030057499999999997,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":3982,"num_read_kb":26490,"num_write":811,"num_write_kb":16305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"147'687","reported_seq":3494,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:20.369085+0000","last_change":"2024-09-16T11:46:20.369043+0000","last_active":"2024-09-16T11:46:20.369085+0000","last_peered":"2024-09-16T11:46:20.369085+0000","last_clean":"2024-09-16T11:46:20.369085+0000","last_became_active":"2024-09-16T11:45:39.641187+0000","last_became_peered":"2024-09-16T11:45:39.641187+0000","last_unstale":"2024-09-16T11:46:20.369085+0000","last_undegraded":"2024-09-16T11:46:20.369085+0000","last_fullsized":"2024-09-16T11:46:20.369085+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'687","last_scrub_stamp":"2024-09-16T11:46:20.368945+0000","last_deep_scrub":"147'687","last_deep_scrub_stamp":"2024-09-16T11:46:20.368945+0000","last_clean_scrub_stamp":"2024-09-16T11:46:20.368945+0000","objects_scrubbed":22,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:48.585782+0000","scrub_duration":0.016326818,"objects_trimmed":2,"snaptrim_duration":0.0074563379999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4363,"num_read_kb":20582,"num_write":914,"num_write_kb":10744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":3197941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"147'569","reported_seq":2545,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:08.145807+0000","last_change":"2024-09-16T11:46:08.145768+0000","last_active":"2024-09-16T11:46:08.145807+0000","last_peered":"2024-09-16T11:46:08.145807+0000","last_clean":"2024-09-16T11:46:08.145807+0000","last_became_active":"2024-09-16T11:43:31.776781+0000","last_became_peered":"2024-09-16T11:43:31.776781+0000","last_unstale":"2024-09-16T11:46:08.145807+0000","last_undegraded":"2024-09-16T11:46:08.145807+0000","last_fullsized":"2024-09-16T11:46:08.145807+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'569","last_scrub_stamp":"2024-09-16T11:46:08.145641+0000","last_deep_scrub":"147'569","last_deep_scrub_stamp":"2024-09-16T11:46:08.145641+0000","last_clean_scrub_stamp":"2024-09-16T11:46:08.145641+0000","objects_scrubbed":23,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:45.614029+0000","scrub_duration":0.01907329,"objects_trimmed":0,"snaptrim_duration":0.00032933499999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3911,"num_read_kb":15179,"num_write":595,"num_write_kb":10620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"147'311","reported_seq":1936,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:37.579949+0000","last_change":"2024-09-16T11:46:37.579904+0000","last_active":"2024-09-16T11:46:37.579949+0000","last_peered":"2024-09-16T11:46:37.579949+0000","last_clean":"2024-09-16T11:46:37.579949+0000","last_became_active":"2024-09-16T11:43:31.780164+0000","last_became_peered":"2024-09-16T11:43:31.780164+0000","last_unstale":"2024-09-16T11:46:37.579949+0000","last_undegraded":"2024-09-16T11:46:37.579949+0000","last_fullsized":"2024-09-16T11:46:37.579949+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'311","last_scrub_stamp":"2024-09-16T11:46:37.579817+0000","last_deep_scrub":"147'311","last_deep_scrub_stamp":"2024-09-16T11:46:37.579817+0000","last_clean_scrub_stamp":"2024-09-16T11:46:37.579817+0000","objects_scrubbed":17,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:08:23.283984+0000","scrub_duration":0.011404694999999999,"objects_trimmed":0,"snaptrim_duration":0.001079981,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":1687,"num_read_kb":10324,"num_write":299,"num_write_kb":6058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"147'844","reported_seq":2312,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:09.382654+0000","last_change":"2024-09-16T11:46:09.382609+0000","last_active":"2024-09-16T11:46:09.382654+0000","last_peered":"2024-09-16T11:46:09.382654+0000","last_clean":"2024-09-16T11:46:09.382654+0000","last_became_active":"2024-09-16T11:45:39.641126+0000","last_became_peered":"2024-09-16T11:45:39.641126+0000","last_unstale":"2024-09-16T11:46:09.382654+0000","last_undegraded":"2024-09-16T11:46:09.382654+0000","last_fullsized":"2024-09-16T11:46:09.382654+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'844","last_scrub_stamp":"2024-09-16T11:46:09.382515+0000","last_deep_scrub":"147'844","last_deep_scrub_stamp":"2024-09-16T11:46:09.382515+0000","last_clean_scrub_stamp":"2024-09-16T11:46:09.382515+0000","objects_scrubbed":40,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:20.750189+0000","scrub_duration":0.029339613,"objects_trimmed":2,"snaptrim_duration":0.0074597470000000001,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":4259,"num_read_kb":18254,"num_write":1055,"num_write_kb":13061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"147'426","reported_seq":1945,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:13.341933+0000","last_change":"2024-09-16T11:46:13.341894+0000","last_active":"2024-09-16T11:46:13.341933+0000","last_peered":"2024-09-16T11:46:13.341933+0000","last_clean":"2024-09-16T11:46:13.341933+0000","last_became_active":"2024-09-16T11:45:39.637293+0000","last_became_peered":"2024-09-16T11:45:39.637293+0000","last_unstale":"2024-09-16T11:46:13.341933+0000","last_undegraded":"2024-09-16T11:46:13.341933+0000","last_fullsized":"2024-09-16T11:46:13.341933+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'426","last_scrub_stamp":"2024-09-16T11:46:13.341770+0000","last_deep_scrub":"147'426","last_deep_scrub_stamp":"2024-09-16T11:46:13.341770+0000","last_clean_scrub_stamp":"2024-09-16T11:46:13.341770+0000","objects_scrubbed":25,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:59.826641+0000","scrub_duration":0.018777958000000001,"objects_trimmed":4,"snaptrim_duration":0.016205368000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3249,"num_read_kb":9637,"num_write":346,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":150,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:10.823125+0000","last_change":"2024-09-16T11:46:10.823082+0000","last_active":"2024-09-16T11:46:10.823125+0000","last_peered":"2024-09-16T11:46:10.823125+0000","last_clean":"2024-09-16T11:46:10.823125+0000","last_became_active":"2024-09-16T11:43:14.731837+0000","last_became_peered":"2024-09-16T11:43:14.731837+0000","last_unstale":"2024-09-16T11:46:10.823125+0000","last_undegraded":"2024-09-16T11:46:10.823125+0000","last_fullsized":"2024-09-16T11:46:10.823125+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:10.822991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:10.822991+0000","last_clean_scrub_stamp":"2024-09-16T11:46:10.822991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:54:42.610189+0000","scrub_duration":0.001279783,"objects_trimmed":0,"snaptrim_duration":0.00035794499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"147'368","reported_seq":1793,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:23.251049+0000","last_change":"2024-09-16T11:46:23.251021+0000","last_active":"2024-09-16T11:46:23.251049+0000","last_peered":"2024-09-16T11:46:23.251049+0000","last_clean":"2024-09-16T11:46:23.251049+0000","last_became_active":"2024-09-16T11:45:47.464115+0000","last_became_peered":"2024-09-16T11:45:47.464115+0000","last_unstale":"2024-09-16T11:46:23.251049+0000","last_undegraded":"2024-09-16T11:46:23.251049+0000","last_fullsized":"2024-09-16T11:46:23.251049+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'368","last_scrub_stamp":"2024-09-16T11:46:23.250916+0000","last_deep_scrub":"147'368","last_deep_scrub_stamp":"2024-09-16T11:46:23.250916+0000","last_clean_scrub_stamp":"2024-09-16T11:46:23.250916+0000","objects_scrubbed":21,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:11.594297+0000","scrub_duration":0.016643457,"objects_trimmed":2,"snaptrim_duration":0.00038392500000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":2491,"num_read_kb":9150,"num_write":283,"num_write_kb":9998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":152,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:10.560106+0000","last_change":"2024-09-16T11:46:10.560058+0000","last_active":"2024-09-16T11:46:10.560106+0000","last_peered":"2024-09-16T11:46:10.560106+0000","last_clean":"2024-09-16T11:46:10.560106+0000","last_became_active":"2024-09-16T11:43:14.731450+0000","last_became_peered":"2024-09-16T11:43:14.731450+0000","last_unstale":"2024-09-16T11:46:10.560106+0000","last_undegraded":"2024-09-16T11:46:10.560106+0000","last_fullsized":"2024-09-16T11:46:10.560106+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:10.559951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:10.559951+0000","last_clean_scrub_stamp":"2024-09-16T11:46:10.559951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:40.196781+0000","scrub_duration":0.0018533320000000001,"objects_trimmed":0,"snaptrim_duration":0.00038809899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"147'500","reported_seq":4615,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:14.383623+0000","last_change":"2024-09-16T11:46:14.383596+0000","last_active":"2024-09-16T11:46:14.383623+0000","last_peered":"2024-09-16T11:46:14.383623+0000","last_clean":"2024-09-16T11:46:14.383623+0000","last_became_active":"2024-09-16T11:45:39.641298+0000","last_became_peered":"2024-09-16T11:45:39.641298+0000","last_unstale":"2024-09-16T11:46:14.383623+0000","last_undegraded":"2024-09-16T11:46:14.383623+0000","last_fullsized":"2024-09-16T11:46:14.383623+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'500","last_scrub_stamp":"2024-09-16T11:46:14.383501+0000","last_deep_scrub":"147'500","last_deep_scrub_stamp":"2024-09-16T11:46:14.383501+0000","last_clean_scrub_stamp":"2024-09-16T11:46:14.383501+0000","objects_scrubbed":23,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:51:05.634633+0000","scrub_duration":0.016287149000000001,"objects_trimmed":6,"snaptrim_duration":0.0094940819999999992,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5544,"num_read_kb":23694,"num_write":515,"num_write_kb":8223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":150,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:11.278804+0000","last_change":"2024-09-16T11:46:11.278740+0000","last_active":"2024-09-16T11:46:11.278804+0000","last_peered":"2024-09-16T11:46:11.278804+0000","last_clean":"2024-09-16T11:46:11.278804+0000","last_became_active":"2024-09-16T11:43:14.743529+0000","last_became_peered":"2024-09-16T11:43:14.743529+0000","last_unstale":"2024-09-16T11:46:11.278804+0000","last_undegraded":"2024-09-16T11:46:11.278804+0000","last_fullsized":"2024-09-16T11:46:11.278804+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:11.278624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:11.278624+0000","last_clean_scrub_stamp":"2024-09-16T11:46:11.278624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:01.706616+0000","scrub_duration":0.0020868729999999999,"objects_trimmed":0,"snaptrim_duration":0.00046617199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"147'646","reported_seq":6897,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:15.574106+0000","last_change":"2024-09-16T11:46:15.574056+0000","last_active":"2024-09-16T11:46:15.574106+0000","last_peered":"2024-09-16T11:46:15.574106+0000","last_clean":"2024-09-16T11:46:15.574106+0000","last_became_active":"2024-09-16T11:43:31.777896+0000","last_became_peered":"2024-09-16T11:43:31.777896+0000","last_unstale":"2024-09-16T11:46:15.574106+0000","last_undegraded":"2024-09-16T11:46:15.574106+0000","last_fullsized":"2024-09-16T11:46:15.574106+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'646","last_scrub_stamp":"2024-09-16T11:46:15.573965+0000","last_deep_scrub":"147'646","last_deep_scrub_stamp":"2024-09-16T11:46:15.573965+0000","last_clean_scrub_stamp":"2024-09-16T11:46:15.573965+0000","objects_scrubbed":19,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:16.367155+0000","scrub_duration":0.014842227,"objects_trimmed":0,"snaptrim_duration":0.0011770719999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":7410,"num_read_kb":47228,"num_write":617,"num_write_kb":8894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":152,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:03.502189+0000","last_change":"2024-09-16T11:46:03.502167+0000","last_active":"2024-09-16T11:46:03.502189+0000","last_peered":"2024-09-16T11:46:03.502189+0000","last_clean":"2024-09-16T11:46:03.502189+0000","last_became_active":"2024-09-16T11:43:14.743564+0000","last_became_peered":"2024-09-16T11:43:14.743564+0000","last_unstale":"2024-09-16T11:46:03.502189+0000","last_undegraded":"2024-09-16T11:46:03.502189+0000","last_fullsized":"2024-09-16T11:46:03.502189+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:03.502083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:03.502083+0000","last_clean_scrub_stamp":"2024-09-16T11:46:03.502083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:51.618689+0000","scrub_duration":0.0011994970000000001,"objects_trimmed":0,"snaptrim_duration":0.00039449400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"147'396","reported_seq":1108,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:29.644126+0000","last_change":"2024-09-16T11:46:29.644091+0000","last_active":"2024-09-16T11:46:29.644126+0000","last_peered":"2024-09-16T11:46:29.644126+0000","last_clean":"2024-09-16T11:46:29.644126+0000","last_became_active":"2024-09-16T11:43:31.778506+0000","last_became_peered":"2024-09-16T11:43:31.778506+0000","last_unstale":"2024-09-16T11:46:29.644126+0000","last_undegraded":"2024-09-16T11:46:29.644126+0000","last_fullsized":"2024-09-16T11:46:29.644126+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'396","last_scrub_stamp":"2024-09-16T11:46:29.644005+0000","last_deep_scrub":"147'396","last_deep_scrub_stamp":"2024-09-16T11:46:29.644005+0000","last_clean_scrub_stamp":"2024-09-16T11:46:29.644005+0000","objects_scrubbed":14,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:28.920272+0000","scrub_duration":0.0097048340000000007,"objects_trimmed":0,"snaptrim_duration":0.0011478079999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1438,"num_read_kb":4478,"num_write":491,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":121,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:12.209849+0000","last_change":"2024-09-16T11:46:12.209821+0000","last_active":"2024-09-16T11:46:12.209849+0000","last_peered":"2024-09-16T11:46:12.209849+0000","last_clean":"2024-09-16T11:46:12.209849+0000","last_became_active":"2024-09-16T11:45:47.462295+0000","last_became_peered":"2024-09-16T11:45:47.462295+0000","last_unstale":"2024-09-16T11:46:12.209849+0000","last_undegraded":"2024-09-16T11:46:12.209849+0000","last_fullsized":"2024-09-16T11:46:12.209849+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:12.209728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:12.209728+0000","last_clean_scrub_stamp":"2024-09-16T11:46:12.209728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:49.726383+0000","scrub_duration":0.001514369,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"147'3","reported_seq":152,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:07.839398+0000","last_change":"2024-09-16T11:46:07.839348+0000","last_active":"2024-09-16T11:46:07.839398+0000","last_peered":"2024-09-16T11:46:07.839398+0000","last_clean":"2024-09-16T11:46:07.839398+0000","last_became_active":"2024-09-16T11:43:14.730764+0000","last_became_peered":"2024-09-16T11:43:14.730764+0000","last_unstale":"2024-09-16T11:46:07.839398+0000","last_undegraded":"2024-09-16T11:46:07.839398+0000","last_fullsized":"2024-09-16T11:46:07.839398+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'3","last_scrub_stamp":"2024-09-16T11:46:07.839246+0000","last_deep_scrub":"147'3","last_deep_scrub_stamp":"2024-09-16T11:46:07.839246+0000","last_clean_scrub_stamp":"2024-09-16T11:46:07.839246+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:53.638295+0000","scrub_duration":0.003326922,"objects_trimmed":0,"snaptrim_duration":0.0010531290000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"147'755","reported_seq":2635,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:35.552978+0000","last_change":"2024-09-16T11:46:35.552940+0000","last_active":"2024-09-16T11:46:35.552978+0000","last_peered":"2024-09-16T11:46:35.552978+0000","last_clean":"2024-09-16T11:46:35.552978+0000","last_became_active":"2024-09-16T11:43:31.777796+0000","last_became_peered":"2024-09-16T11:43:31.777796+0000","last_unstale":"2024-09-16T11:46:35.552978+0000","last_undegraded":"2024-09-16T11:46:35.552978+0000","last_fullsized":"2024-09-16T11:46:35.552978+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'755","last_scrub_stamp":"2024-09-16T11:46:35.552808+0000","last_deep_scrub":"147'755","last_deep_scrub_stamp":"2024-09-16T11:46:35.552808+0000","last_clean_scrub_stamp":"2024-09-16T11:46:35.552808+0000","objects_scrubbed":21,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:51:45.388077+0000","scrub_duration":0.016738751,"objects_trimmed":0,"snaptrim_duration":0.00051098400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4214,"num_read_kb":15199,"num_write":913,"num_write_kb":10139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:19.703058+0000","last_change":"2024-09-16T11:46:19.703033+0000","last_active":"2024-09-16T11:46:19.703058+0000","last_peered":"2024-09-16T11:46:19.703058+0000","last_clean":"2024-09-16T11:46:19.703058+0000","last_became_active":"2024-09-16T11:43:22.439843+0000","last_became_peered":"2024-09-16T11:43:22.439843+0000","last_unstale":"2024-09-16T11:46:19.703058+0000","last_undegraded":"2024-09-16T11:46:19.703058+0000","last_fullsized":"2024-09-16T11:46:19.703058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:19.702943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:19.702943+0000","last_clean_scrub_stamp":"2024-09-16T11:46:19.702943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:12:02.946224+0000","scrub_duration":0.001388046,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":169,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:04.489635+0000","last_change":"2024-09-16T11:46:04.489617+0000","last_active":"2024-09-16T11:46:04.489635+0000","last_peered":"2024-09-16T11:46:04.489635+0000","last_clean":"2024-09-16T11:46:04.489635+0000","last_became_active":"2024-09-16T11:45:39.635334+0000","last_became_peered":"2024-09-16T11:45:39.635334+0000","last_unstale":"2024-09-16T11:46:04.489635+0000","last_undegraded":"2024-09-16T11:46:04.489635+0000","last_fullsized":"2024-09-16T11:46:04.489635+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:04.489545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:04.489545+0000","last_clean_scrub_stamp":"2024-09-16T11:46:04.489545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:38.788913+0000","scrub_duration":0.00112024,"objects_trimmed":0,"snaptrim_duration":0.00034766000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"147'569","reported_seq":3424,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:22.617289+0000","last_change":"2024-09-16T11:46:22.617240+0000","last_active":"2024-09-16T11:46:22.617289+0000","last_peered":"2024-09-16T11:46:22.617289+0000","last_clean":"2024-09-16T11:46:22.617289+0000","last_became_active":"2024-09-16T11:43:31.777967+0000","last_became_peered":"2024-09-16T11:43:31.777967+0000","last_unstale":"2024-09-16T11:46:22.617289+0000","last_undegraded":"2024-09-16T11:46:22.617289+0000","last_fullsized":"2024-09-16T11:46:22.617289+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'569","last_scrub_stamp":"2024-09-16T11:46:22.617140+0000","last_deep_scrub":"147'569","last_deep_scrub_stamp":"2024-09-16T11:46:22.617140+0000","last_clean_scrub_stamp":"2024-09-16T11:46:22.617140+0000","objects_scrubbed":23,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:17:13.197926+0000","scrub_duration":0.016575501999999999,"objects_trimmed":0,"snaptrim_duration":0.00081483900000000004,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":3645,"num_read_kb":23388,"num_write":630,"num_write_kb":9618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"147'5","reported_seq":147,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:11.474783+0000","last_change":"2024-09-16T11:46:11.474728+0000","last_active":"2024-09-16T11:46:11.474783+0000","last_peered":"2024-09-16T11:46:11.474783+0000","last_clean":"2024-09-16T11:46:11.474783+0000","last_became_active":"2024-09-16T11:43:22.437666+0000","last_became_peered":"2024-09-16T11:43:22.437666+0000","last_unstale":"2024-09-16T11:46:11.474783+0000","last_undegraded":"2024-09-16T11:46:11.474783+0000","last_fullsized":"2024-09-16T11:46:11.474783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'4","last_scrub_stamp":"2024-09-16T11:46:11.474608+0000","last_deep_scrub":"147'4","last_deep_scrub_stamp":"2024-09-16T11:46:11.474608+0000","last_clean_scrub_stamp":"2024-09-16T11:46:11.474608+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:09:01.168170+0000","scrub_duration":0.0038928880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":166,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":150,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:07.323759+0000","last_change":"2024-09-16T11:46:07.323734+0000","last_active":"2024-09-16T11:46:07.323759+0000","last_peered":"2024-09-16T11:46:07.323759+0000","last_clean":"2024-09-16T11:46:07.323759+0000","last_became_active":"2024-09-16T11:43:14.730546+0000","last_became_peered":"2024-09-16T11:43:14.730546+0000","last_unstale":"2024-09-16T11:46:07.323759+0000","last_undegraded":"2024-09-16T11:46:07.323759+0000","last_fullsized":"2024-09-16T11:46:07.323759+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:46:07.323641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:46:07.323641+0000","last_clean_scrub_stamp":"2024-09-16T11:46:07.323641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:09.437604+0000","scrub_duration":0.0020561540000000001,"objects_trimmed":0,"snaptrim_duration":0.00046014899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"147'427","reported_seq":1801,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:17.591254+0000","last_change":"2024-09-16T11:46:17.591209+0000","last_active":"2024-09-16T11:46:17.591254+0000","last_peered":"2024-09-16T11:46:17.591254+0000","last_clean":"2024-09-16T11:46:17.591254+0000","last_became_active":"2024-09-16T11:43:31.776645+0000","last_became_peered":"2024-09-16T11:43:31.776645+0000","last_unstale":"2024-09-16T11:46:17.591254+0000","last_undegraded":"2024-09-16T11:46:17.591254+0000","last_fullsized":"2024-09-16T11:46:17.591254+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'427","last_scrub_stamp":"2024-09-16T11:46:17.591116+0000","last_deep_scrub":"147'427","last_deep_scrub_stamp":"2024-09-16T11:46:17.591116+0000","last_clean_scrub_stamp":"2024-09-16T11:46:17.591116+0000","objects_scrubbed":20,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:55.287547+0000","scrub_duration":0.013905288,"objects_trimmed":0,"snaptrim_duration":0.00066659399999999995,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1650,"num_read_kb":12114,"num_write":331,"num_write_kb":15245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"147'6","reported_seq":111,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:06.411314+0000","last_change":"2024-09-16T11:46:06.411284+0000","last_active":"2024-09-16T11:46:06.411314+0000","last_peered":"2024-09-16T11:46:06.411314+0000","last_clean":"2024-09-16T11:46:06.411314+0000","last_became_active":"2024-09-16T11:45:39.637629+0000","last_became_peered":"2024-09-16T11:45:39.637629+0000","last_unstale":"2024-09-16T11:46:06.411314+0000","last_undegraded":"2024-09-16T11:46:06.411314+0000","last_fullsized":"2024-09-16T11:46:06.411314+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'6","last_scrub_stamp":"2024-09-16T11:46:06.411204+0000","last_deep_scrub":"147'6","last_deep_scrub_stamp":"2024-09-16T11:46:06.411204+0000","last_clean_scrub_stamp":"2024-09-16T11:46:06.411204+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:03.945238+0000","scrub_duration":0.0038620859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"147'2","reported_seq":151,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:10.884409+0000","last_change":"2024-09-16T11:46:10.884380+0000","last_active":"2024-09-16T11:46:10.884409+0000","last_peered":"2024-09-16T11:46:10.884409+0000","last_clean":"2024-09-16T11:46:10.884409+0000","last_became_active":"2024-09-16T11:43:14.727831+0000","last_became_peered":"2024-09-16T11:43:14.727831+0000","last_unstale":"2024-09-16T11:46:10.884409+0000","last_undegraded":"2024-09-16T11:46:10.884409+0000","last_fullsized":"2024-09-16T11:46:10.884409+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'2","last_scrub_stamp":"2024-09-16T11:46:10.884282+0000","last_deep_scrub":"147'2","last_deep_scrub_stamp":"2024-09-16T11:46:10.884282+0000","last_clean_scrub_stamp":"2024-09-16T11:46:10.884282+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:38.365702+0000","scrub_duration":0.0031331610000000002,"objects_trimmed":0,"snaptrim_duration":0.00039305600000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"147'74","reported_seq":239,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:06.277326+0000","last_change":"2024-09-16T11:46:06.277299+0000","last_active":"2024-09-16T11:46:06.277326+0000","last_peered":"2024-09-16T11:46:06.277326+0000","last_clean":"2024-09-16T11:46:06.277326+0000","last_became_active":"2024-09-16T11:43:12.701759+0000","last_became_peered":"2024-09-16T11:43:12.701759+0000","last_unstale":"2024-09-16T11:46:06.277326+0000","last_undegraded":"2024-09-16T11:46:06.277326+0000","last_fullsized":"2024-09-16T11:46:06.277326+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'74","last_scrub_stamp":"2024-09-16T11:46:06.277205+0000","last_deep_scrub":"147'74","last_deep_scrub_stamp":"2024-09-16T11:46:06.277205+0000","last_clean_scrub_stamp":"2024-09-16T11:46:06.277205+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:54.267542+0000","scrub_duration":0.0041307949999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"147'544","reported_seq":3470,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:11.844641+0000","last_change":"2024-09-16T11:46:11.844611+0000","last_active":"2024-09-16T11:46:11.844641+0000","last_peered":"2024-09-16T11:46:11.844641+0000","last_clean":"2024-09-16T11:46:11.844641+0000","last_became_active":"2024-09-16T11:43:31.782071+0000","last_became_peered":"2024-09-16T11:43:31.782071+0000","last_unstale":"2024-09-16T11:46:11.844641+0000","last_undegraded":"2024-09-16T11:46:11.844641+0000","last_fullsized":"2024-09-16T11:46:11.844641+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'544","last_scrub_stamp":"2024-09-16T11:46:11.844514+0000","last_deep_scrub":"147'544","last_deep_scrub_stamp":"2024-09-16T11:46:11.844514+0000","last_clean_scrub_stamp":"2024-09-16T11:46:11.844514+0000","objects_scrubbed":21,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:10.626670+0000","scrub_duration":0.01463768,"objects_trimmed":0,"snaptrim_duration":0.000599604,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3801,"num_read_kb":24361,"num_write":540,"num_write_kb":7978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"147'697","reported_seq":9050,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:41.573261+0000","last_change":"2024-09-16T11:46:41.573222+0000","last_active":"2024-09-16T11:46:41.573261+0000","last_peered":"2024-09-16T11:46:41.573261+0000","last_clean":"2024-09-16T11:46:41.573261+0000","last_became_active":"2024-09-16T11:44:10.920131+0000","last_became_peered":"2024-09-16T11:44:10.920131+0000","last_unstale":"2024-09-16T11:46:41.573261+0000","last_undegraded":"2024-09-16T11:46:41.573261+0000","last_fullsized":"2024-09-16T11:46:41.573261+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'697","last_scrub_stamp":"2024-09-16T11:46:41.573121+0000","last_deep_scrub":"147'697","last_deep_scrub_stamp":"2024-09-16T11:46:41.573121+0000","last_clean_scrub_stamp":"2024-09-16T11:46:41.573121+0000","objects_scrubbed":32,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:30:15.083795+0000","scrub_duration":0.019439708999999999,"objects_trimmed":0,"snaptrim_duration":0.00082161099999999998,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":9498,"num_read_kb":69976,"num_write":712,"num_write_kb":19149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":8960357,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"147'904","reported_seq":3899,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:19.178394+0000","last_change":"2024-09-16T11:46:19.178350+0000","last_active":"2024-09-16T11:46:19.178394+0000","last_peered":"2024-09-16T11:46:19.178394+0000","last_clean":"2024-09-16T11:46:19.178394+0000","last_became_active":"2024-09-16T11:45:47.464353+0000","last_became_peered":"2024-09-16T11:45:47.464353+0000","last_unstale":"2024-09-16T11:46:19.178394+0000","last_undegraded":"2024-09-16T11:46:19.178394+0000","last_fullsized":"2024-09-16T11:46:19.178394+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'904","last_scrub_stamp":"2024-09-16T11:46:19.178262+0000","last_deep_scrub":"147'904","last_deep_scrub_stamp":"2024-09-16T11:46:19.178262+0000","last_clean_scrub_stamp":"2024-09-16T11:46:19.178262+0000","objects_scrubbed":26,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:10:26.656285+0000","scrub_duration":0.020921347,"objects_trimmed":0,"snaptrim_duration":0.001068741,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":8005,"num_read_kb":12353,"num_write":894,"num_write_kb":6127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"147'655","reported_seq":1861,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:15.836157+0000","last_change":"2024-09-16T11:46:15.836120+0000","last_active":"2024-09-16T11:46:15.836157+0000","last_peered":"2024-09-16T11:46:15.836157+0000","last_clean":"2024-09-16T11:46:15.836157+0000","last_became_active":"2024-09-16T11:43:31.779384+0000","last_became_peered":"2024-09-16T11:43:31.779384+0000","last_unstale":"2024-09-16T11:46:15.836157+0000","last_undegraded":"2024-09-16T11:46:15.836157+0000","last_fullsized":"2024-09-16T11:46:15.836157+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'655","last_scrub_stamp":"2024-09-16T11:46:15.836039+0000","last_deep_scrub":"147'655","last_deep_scrub_stamp":"2024-09-16T11:46:15.836039+0000","last_clean_scrub_stamp":"2024-09-16T11:46:15.836039+0000","objects_scrubbed":32,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:22:30.289390+0000","scrub_duration":0.021988002999999999,"objects_trimmed":0,"snaptrim_duration":0.000647592,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":2523,"num_read_kb":9199,"num_write":666,"num_write_kb":6203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"147'399","reported_seq":3413,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:25.451721+0000","last_change":"2024-09-16T11:46:25.451680+0000","last_active":"2024-09-16T11:46:25.451721+0000","last_peered":"2024-09-16T11:46:25.451721+0000","last_clean":"2024-09-16T11:46:25.451721+0000","last_became_active":"2024-09-16T11:43:31.775335+0000","last_became_peered":"2024-09-16T11:43:31.775335+0000","last_unstale":"2024-09-16T11:46:25.451721+0000","last_undegraded":"2024-09-16T11:46:25.451721+0000","last_fullsized":"2024-09-16T11:46:25.451721+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'399","last_scrub_stamp":"2024-09-16T11:46:25.451584+0000","last_deep_scrub":"147'399","last_deep_scrub_stamp":"2024-09-16T11:46:25.451584+0000","last_clean_scrub_stamp":"2024-09-16T11:46:25.451584+0000","objects_scrubbed":22,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:04:23.569454+0000","scrub_duration":0.014250751000000001,"objects_trimmed":0,"snaptrim_duration":0.0011352389999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4122,"num_read_kb":15191,"num_write":417,"num_write_kb":10211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"147'545","reported_seq":2303,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:20.171393+0000","last_change":"2024-09-16T11:46:20.171358+0000","last_active":"2024-09-16T11:46:20.171393+0000","last_peered":"2024-09-16T11:46:20.171393+0000","last_clean":"2024-09-16T11:46:20.171393+0000","last_became_active":"2024-09-16T11:45:47.463487+0000","last_became_peered":"2024-09-16T11:45:47.463487+0000","last_unstale":"2024-09-16T11:46:20.171393+0000","last_undegraded":"2024-09-16T11:46:20.171393+0000","last_fullsized":"2024-09-16T11:46:20.171393+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'545","last_scrub_stamp":"2024-09-16T11:46:20.171264+0000","last_deep_scrub":"147'545","last_deep_scrub_stamp":"2024-09-16T11:46:20.171264+0000","last_clean_scrub_stamp":"2024-09-16T11:46:20.171264+0000","objects_scrubbed":24,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:05:39.743556+0000","scrub_duration":0.015237156999999999,"objects_trimmed":2,"snaptrim_duration":0.001754511,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":4037,"num_read_kb":5871,"num_write":454,"num_write_kb":2926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"147'639","reported_seq":5107,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:27.283861+0000","last_change":"2024-09-16T11:46:27.283829+0000","last_active":"2024-09-16T11:46:27.283861+0000","last_peered":"2024-09-16T11:46:27.283861+0000","last_clean":"2024-09-16T11:46:27.283861+0000","last_became_active":"2024-09-16T11:45:47.461210+0000","last_became_peered":"2024-09-16T11:45:47.461210+0000","last_unstale":"2024-09-16T11:46:27.283861+0000","last_undegraded":"2024-09-16T11:46:27.283861+0000","last_fullsized":"2024-09-16T11:46:27.283861+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'639","last_scrub_stamp":"2024-09-16T11:46:27.283710+0000","last_deep_scrub":"147'639","last_deep_scrub_stamp":"2024-09-16T11:46:27.283710+0000","last_clean_scrub_stamp":"2024-09-16T11:46:27.283710+0000","objects_scrubbed":23,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:58.937088+0000","scrub_duration":0.018474912,"objects_trimmed":4,"snaptrim_duration":0.00043040400000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6463,"num_read_kb":36386,"num_write":811,"num_write_kb":14056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"147'628","reported_seq":1965,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:11.204020+0000","last_change":"2024-09-16T11:46:11.203985+0000","last_active":"2024-09-16T11:46:11.204020+0000","last_peered":"2024-09-16T11:46:11.204020+0000","last_clean":"2024-09-16T11:46:11.204020+0000","last_became_active":"2024-09-16T11:45:47.464052+0000","last_became_peered":"2024-09-16T11:45:47.464052+0000","last_unstale":"2024-09-16T11:46:11.204020+0000","last_undegraded":"2024-09-16T11:46:11.204020+0000","last_fullsized":"2024-09-16T11:46:11.204020+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'628","last_scrub_stamp":"2024-09-16T11:46:11.203896+0000","last_deep_scrub":"147'628","last_deep_scrub_stamp":"2024-09-16T11:46:11.203896+0000","last_clean_scrub_stamp":"2024-09-16T11:46:11.203896+0000","objects_scrubbed":23,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:36.692499+0000","scrub_duration":0.016278948000000001,"objects_trimmed":0,"snaptrim_duration":0.00059308799999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2764,"num_read_kb":9887,"num_write":657,"num_write_kb":5333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"147'429","reported_seq":2149,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-16T11:46:39.607833+0000","last_change":"2024-09-16T11:46:39.607805+0000","last_active":"2024-09-16T11:46:39.607833+0000","last_peered":"2024-09-16T11:46:39.607833+0000","last_clean":"2024-09-16T11:46:39.607833+0000","last_became_active":"2024-09-16T11:45:39.643207+0000","last_became_peered":"2024-09-16T11:45:39.643207+0000","last_unstale":"2024-09-16T11:46:39.607833+0000","last_undegraded":"2024-09-16T11:46:39.607833+0000","last_fullsized":"2024-09-16T11:46:39.607833+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'429","last_scrub_stamp":"2024-09-16T11:46:39.607717+0000","last_deep_scrub":"147'429","last_deep_scrub_stamp":"2024-09-16T11:46:39.607717+0000","last_clean_scrub_stamp":"2024-09-16T11:46:39.607717+0000","objects_scrubbed":21,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:50.891949+0000","scrub_duration":0.015712501,"objects_trimmed":0,"snaptrim_duration":0.0070987940000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":3609,"num_read_kb":5161,"num_write":362,"num_write_kb":8941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":728,"num_read":135475,"num_read_kb":642896,"num_write":18060,"num_write_kb":290606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":12158298,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},"log_size":17219,"ondisk_log_size":17219,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":251,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":251,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10312,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299124,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072716,"statfs":{"total":96636764160,"available":96330461184,"internally_reserved":0,"allocated":1814528,"data_stored":1481880,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.46500000000000002,"15min":0.45100000000000001},"min":{"1min":0.26400000000000001,"5min":0.182,"15min":0.182},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.497,"15min":0.48199999999999998},"min":{"1min":0.33100000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64600000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.61799999999999999},"min":{"1min":0.42299999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.91900000000000004,"5min":4.3559999999999999,"15min":4.3559999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.46400000000000002,"15min":0.44400000000000001},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52500000000000002}]},{"osd":2,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.38,"5min":0.248,"15min":0.248},"max":{"1min":0.68500000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.437,"5min":0.309,"15min":0.309},"max":{"1min":0.86399999999999999,"5min":1.532,"15min":1.532},"last":0.54100000000000004}]},{"osd":3,"last update":"Mon Sep 16 11:46:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.41299999999999998}]},{"osd":4,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.435,"15min":0.44500000000000001},"min":{"1min":0.32900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.65100000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.5},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.45700000000000002},"min":{"1min":0.28000000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.57299999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.35999999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.432,"15min":0.438},"min":{"1min":0.32000000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48899999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.377},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.47299999999999998,"15min":0.48299999999999998},"min":{"1min":0.33600000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.59199999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.42199999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.48299999999999998,"15min":0.47399999999999998},"min":{"1min":0.38700000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47099999999999997,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61699999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48899999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542189,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072760,"statfs":{"total":96636764160,"available":96330506240,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.33300000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66900000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.315,"5min":0.252,"15min":0.252},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.55700000000000005}]},{"osd":1,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45200000000000001,"15min":0.44800000000000001},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.45300000000000001,"15min":0.44700000000000001},"min":{"1min":0.39000000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65800000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.46800000000000003,"15min":0.46100000000000002},"min":{"1min":0.38200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.36799999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69499999999999995,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.81999999999999995}]},{"osd":3,"last update":"Mon Sep 16 11:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.66500000000000004}]},{"osd":4,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.309,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63600000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.45500000000000002},"min":{"1min":0.34499999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.44800000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.45100000000000001,"15min":0.45900000000000002},"min":{"1min":0.29099999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64600000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.46500000000000002,"15min":0.46899999999999997},"min":{"1min":0.36799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.47299999999999998}]},{"osd":7,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.34499999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.501,"15min":0.51100000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.64800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53400000000000003}]}]},{"osd":3,"up_from":131,"seq":562640715791,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302152,"kb_used_data":1728,"kb_used_omap":43,"kb_used_meta":300372,"kb_avail":94069688,"statfs":{"total":96636764160,"available":96327360512,"internally_reserved":0,"allocated":1769472,"data_stored":892524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44228,"internal_metadata":307581756},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.505}]},{"osd":1,"last update":"Mon Sep 16 11:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.52300000000000002}]},{"osd":2,"last update":"Mon Sep 16 11:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.48299999999999998}]},{"osd":4,"last update":"Mon Sep 16 11:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Sep 16 11:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.79400000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.56299999999999994}]},{"osd":7,"last update":"Mon Sep 16 11:46:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.81100000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299076,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072764,"statfs":{"total":96636764160,"available":96330510336,"internally_reserved":0,"allocated":1765376,"data_stored":892439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.33200000000000002,"15min":0.32700000000000001},"min":{"1min":0.20699999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.48999999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.495,"15min":0.39700000000000002},"min":{"1min":0.219,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.60099999999999998,"5min":11.218,"15min":11.218},"last":0.49399999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.32400000000000001},"min":{"1min":0.26000000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.50700000000000001,"5min":0.746,"15min":0.746},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.45200000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Mon Sep 16 11:46:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.439}]},{"osd":4,"last update":"Mon Sep 16 11:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.47599999999999998,"15min":0.47199999999999998},"min":{"1min":0.34100000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.318,"5min":0.315,"15min":0.315},"max":{"1min":0.81100000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.59399999999999997}]},{"osd":5,"last update":"Mon Sep 16 11:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85899999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.41999999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76900000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.621}]},{"osd":6,"last update":"Mon Sep 16 11:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.628},{"interface":"front","average":{"1min":0.622,"5min":0.54000000000000004,"15min":0.53000000000000003},"min":{"1min":0.441,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.71099999999999997}]},{"osd":7,"last update":"Mon Sep 16 11:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.436,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91700000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.77900000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311548,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94060292,"statfs":{"total":96636764160,"available":96317739008,"internally_reserved":0,"allocated":1757184,"data_stored":891472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.45500000000000002,"15min":0.45600000000000002},"min":{"1min":0.379,"5min":0.223,"15min":0.223},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.439,"15min":0.441},"min":{"1min":0.35499999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.48599999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73699999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.41299999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.63200000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.503}]},{"osd":2,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.42999999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63700000000000001,"5min":1.026,"15min":1.026},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.38900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.80000000000000004,"5min":1.01,"15min":1.01},"last":0.45800000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.54900000000000004}]},{"osd":4,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.40400000000000003},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54900000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.46500000000000002,"15min":0.46400000000000002},"min":{"1min":0.39000000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70699999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.432}]},{"osd":5,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44800000000000001,"15min":0.46400000000000002},"min":{"1min":0.315,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60999999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.435},"min":{"1min":0.35099999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.33200000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.746,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48499999999999999,"15min":0.496},"min":{"1min":0.34799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.77300000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.44500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303244,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068596,"statfs":{"total":96636764160,"available":96326242304,"internally_reserved":0,"allocated":1773568,"data_stored":892605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48799999999999999,"15min":0.44500000000000001},"min":{"1min":0.31900000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.64500000000000002,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.503},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.46000000000000002,"15min":0.45500000000000002},"min":{"1min":0.374,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.60699999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.61899999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.47599999999999998,"15min":0.47099999999999997},"min":{"1min":0.32600000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.36299999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.59199999999999997}]},{"osd":2,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.49299999999999999,"15min":0.498},"min":{"1min":0.39400000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.501,"15min":0.503},"min":{"1min":0.38400000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66500000000000004}]},{"osd":3,"last update":"Mon Sep 16 11:46:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.55000000000000004}]},{"osd":5,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41699999999999998,"15min":0.433},"min":{"1min":0.27400000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.628,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.442},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.45200000000000001,"15min":0.46800000000000003},"min":{"1min":0.32800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.76400000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45700000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.47899999999999998},"min":{"1min":0.29999999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66600000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.55000000000000004},"min":{"1min":0.38600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51500000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.50600000000000001},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.79900000000000004,"5min":0.873,"15min":0.873},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.52300000000000002},"min":{"1min":0.34200000000000003,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77900000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.47999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299072,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072768,"statfs":{"total":96636764160,"available":96330514432,"internally_reserved":0,"allocated":1761280,"data_stored":892420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.308,"15min":0.30399999999999999},"min":{"1min":0.20599999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.308},{"interface":"front","average":{"1min":0.377,"5min":0.33900000000000002,"15min":0.32900000000000001},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.37},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.35699999999999998,"15min":0.35199999999999998},"min":{"1min":0.23599999999999999,"5min":0.192,"15min":0.192},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.34200000000000003}]},{"osd":3,"last update":"Mon Sep 16 11:46:38 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.32500000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.58199999999999996,"15min":0.55500000000000005},"min":{"1min":0.28899999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.627}]},{"osd":5,"last update":"Mon Sep 16 11:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33500000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2889999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.61499999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.67800000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.39900000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.2689999999999999,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.58899999999999997,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.65700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299136,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072704,"statfs":{"total":96636764160,"available":96330448896,"internally_reserved":0,"allocated":1826816,"data_stored":1482954,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.39800000000000002,"15min":0.33800000000000002},"min":{"1min":0.19500000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.503,"5min":8.5050000000000008,"15min":8.5050000000000008},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.32900000000000001,"15min":0.32300000000000001},"min":{"1min":0.23799999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.48599999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.40999999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.40000000000000002,"15min":0.33700000000000002},"min":{"1min":0.25,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.50700000000000001,"5min":8.4689999999999994,"15min":8.4689999999999994},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.32600000000000001,"15min":0.32700000000000001},"min":{"1min":0.22900000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.44600000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.313}]},{"osd":3,"last update":"Mon Sep 16 11:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":4.2030000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.37}]},{"osd":4,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.52200000000000002},"min":{"1min":0.45400000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.746,"5min":8.5299999999999994,"15min":8.5299999999999994},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52200000000000002,"15min":0.51200000000000001},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75700000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.48699999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66400000000000003,"15min":0.60799999999999998},"min":{"1min":0.41399999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.125,"5min":8.6039999999999992,"15min":8.6039999999999992},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.38100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.441}]},{"osd":6,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57299999999999995,"15min":0.57399999999999995},"min":{"1min":0.42999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.505},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69099999999999995,"15min":0.63300000000000001},"min":{"1min":0.41099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.157,"5min":8.5809999999999995,"15min":8.5809999999999995},"last":0.42199999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.39900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.625,"15min":0.55700000000000005},"min":{"1min":0.42399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73499999999999999,"5min":8.5570000000000004,"15min":8.5570000000000004},"last":0.498}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5156,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19003,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:46:46.504 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T11:46:46.784 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T11:46:46.784 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T11:46:46.785 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T11:46:46.785 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:46:46.825 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T11:46:46.825 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T11:46:46.825 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:46:46.892 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T11:46:46.893 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T11:46:46.893 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:46:46.971 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T11:46:46.971 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T11:46:46.971 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:46:47.052 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T11:46:47.052 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T11:46:47.052 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:46:47.091 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T11:46:47.092 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T11:46:47.092 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:46:47.175 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T11:46:47.176 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T11:46:47.176 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:46:47.210 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T11:46:47.210 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-16T11:46:47.211 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:46:47.284 INFO:tasks.ceph.osd.7:Stopped 2024-09-16T11:46:47.284 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T11:46:47.284 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-16T11:46:47.284 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:46:47.308 INFO:tasks.ceph.mgr.y:Stopped 2024-09-16T11:46:47.308 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T11:46:47.308 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:46:47.351 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T11:46:47.351 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T11:46:47.351 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T11:46:47.351 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:46:47.362 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T11:46:47.362 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T11:46:47.362 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:46:47.413 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T11:46:47.413 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T11:46:47.413 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:46:47.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-16T11:46:47.426 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T11:46:47.427 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T11:46:47.427 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-09-16T11:46:47.490 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T11:46:47.491 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T11:46:47.730 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T11:46:47.730 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T11:46:47.812 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T11:46:47.812 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T11:46:47.909 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T11:46:47.909 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T11:46:47.969 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-16T11:46:47.969 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-16T11:46:48.111 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-16T11:46:48.111 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-16T11:46:48.183 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-16T11:46:48.183 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-16T11:46:48.240 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-16T11:46:48.240 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-16T11:46:48.290 INFO:tasks.ceph:Archiving mon data... 2024-09-16T11:46:48.291 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907206/data/mon.a.tgz 2024-09-16T11:46:48.291 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-16T11:46:48.306 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.yB8geq7Y0K 2024-09-16T11:46:48.307 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.yB8geq7Y0K 2024-09-16T11:46:48.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.yB8geq7Y0K 2024-09-16T11:46:48.547 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.yB8geq7Y0K is 374KB 2024-09-16T11:46:48.643 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.yB8geq7Y0K 2024-09-16T11:46:48.657 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907206/data/mon.c.tgz 2024-09-16T11:46:48.658 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-16T11:46:48.712 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.PrgfNs2gNv 2024-09-16T11:46:48.712 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.PrgfNs2gNv 2024-09-16T11:46:48.879 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.PrgfNs2gNv 2024-09-16T11:46:48.962 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.PrgfNs2gNv is 512KB 2024-09-16T11:46:49.060 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.PrgfNs2gNv 2024-09-16T11:46:49.075 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907206/data/mon.b.tgz 2024-09-16T11:46:49.075 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-09-16T11:46:49.094 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.aI4Qvj5eNh 2024-09-16T11:46:49.094 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.aI4Qvj5eNh 2024-09-16T11:46:49.278 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.aI4Qvj5eNh 2024-09-16T11:46:49.362 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.aI4Qvj5eNh is 512KB 2024-09-16T11:46:49.461 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.aI4Qvj5eNh 2024-09-16T11:46:49.476 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T11:46:49.476 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T11:46:49.478 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T11:46:49.540 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-16T11:46:49.540 DEBUG:teuthology.orchestra.run.smithi070:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T11:46:49.544 DEBUG:teuthology.orchestra.run.smithi192:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T11:46:49.568 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T11:46:49.606 INFO:teuthology.orchestra.run.smithi192.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T11:46:49.610 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T11:46:49.611 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/crash to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907206/remote/smithi070/crash 2024-09-16T11:46:49.612 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T11:46:49.638 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/crash to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907206/remote/smithi192/crash 2024-09-16T11:46:49.639 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T11:46:49.674 INFO:tasks.ceph:Compressing logs... 2024-09-16T11:46:49.674 DEBUG:teuthology.orchestra.run.smithi070:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T11:46:49.682 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T11:46:49.704 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-16T11:46:49.704 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61271.log 2024-09-16T11:46:49.755 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-16T11:46:49.755 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33152.log 2024-09-16T11:46:49.755 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33232.log 2024-09-16T11:46:49.755 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33152.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.33152.log.gz 2024-09-16T11:46:49.755 INFO:teuthology.orchestra.run.smithi192.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-16T11:46:49.756 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33232.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T11:46:49.756 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33232.log.gz 2024-09-16T11:46:49.756 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33032.log 2024-09-16T11:46:49.756 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33192.log 2024-09-16T11:46:49.756 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33032.log.gz 2024-09-16T11:46:49.756 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33072.log 2024-09-16T11:46:49.756 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32992.log 2024-09-16T11:46:49.756 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33192.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33192.log.gz 2024-09-16T11:46:49.756 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33072.log.gz 2024-09-16T11:46:49.756 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32945.log 2024-09-16T11:46:49.756 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-16T11:46:49.756 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.32992.log: 88.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T11:46:49.756 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32992.log.gz 2024-09-16T11:46:49.757 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.32945.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-16T11:46:49.757 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32945.log.gz 2024-09-16T11:46:49.757 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T11:46:49.757 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-16T11:46:49.757 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-16T11:46:49.757 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33112.log 2024-09-16T11:46:49.757 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.33112.log: 83.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T11:46:49.757 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33112.log.gz 2024-09-16T11:46:49.757 INFO:teuthology.orchestra.run.smithi192.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-16T11:46:49.820 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37382.log 2024-09-16T11:46:49.821 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.61271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59465.log 2024-09-16T11:46:49.821 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61271.log.gz 2024-09-16T11:46:49.821 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33361.log 2024-09-16T11:46:49.821 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37382.log.gz 2024-09-16T11:46:49.821 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59465.log.gz 2024-09-16T11:46:49.821 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38346.log 2024-09-16T11:46:49.821 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60067.log 2024-09-16T11:46:49.821 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33361.log.gz 2024-09-16T11:46:49.821 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38346.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38346.log.gz -5 2024-09-16T11:46:49.821 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44975.log 2024-09-16T11:46:49.821 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38558.log 2024-09-16T11:46:49.821 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60067.log.gz 2024-09-16T11:46:49.822 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.44975.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44975.log.gz 2024-09-16T11:46:49.822 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48753.log 2024-09-16T11:46:49.822 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39371.log 2024-09-16T11:46:49.822 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38558.log.gz 2024-09-16T11:46:49.822 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59507.log 2024-09-16T11:46:49.822 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.48753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48753.log.gz 2024-09-16T11:46:49.822 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38966.log 2024-09-16T11:46:49.822 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39371.log.gz 2024-09-16T11:46:49.822 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37728.log 2024-09-16T11:46:49.822 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59507.log.gz 2024-09-16T11:46:49.822 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.38966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49401.log 2024-09-16T11:46:49.822 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49074.log/var/log/ceph/ceph-client.admin.37728.log: 2024-09-16T11:46:49.822 INFO:teuthology.orchestra.run.smithi070.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.0.38966.log.gz 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37728.log.gz 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.49401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37391.log 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49401.log.gz 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.49074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60599.log 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49074.log.gz 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40673.log 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37391.log.gz 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49837.log 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60599.log.gz 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.40673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46295.log 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40673.log.gz 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.49837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62111.log 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49837.log.gz 2024-09-16T11:46:49.823 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.46295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40974.log 2024-09-16T11:46:49.824 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46295.log.gz 2024-09-16T11:46:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.62111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41990.log 2024-09-16T11:46:49.824 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62111.log.gz 2024-09-16T11:46:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.40974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37394.log 2024-09-16T11:46:49.824 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40974.log.gz 2024-09-16T11:46:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.41990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46994.log 2024-09-16T11:46:49.824 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41990.log.gz 2024-09-16T11:46:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61901.log 2024-09-16T11:46:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37394.log.gz 2024-09-16T11:46:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.46994.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.46994.log.gz -5 2024-09-16T11:46:49.824 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60361.log 2024-09-16T11:46:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61901.log.gz 2024-09-16T11:46:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27208.log 2024-09-16T11:46:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48415.log 2024-09-16T11:46:49.825 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60361.log.gz 2024-09-16T11:46:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.tmp-client.admin.27208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60435.log 2024-09-16T11:46:49.825 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27208.log.gz 2024-09-16T11:46:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.48415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37392.log 2024-09-16T11:46:49.825 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48415.log.gz 2024-09-16T11:46:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60435.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60435.log.gz -5 2024-09-16T11:46:49.825 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37965.log 2024-09-16T11:46:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60765.log 2024-09-16T11:46:49.825 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37392.log.gz 2024-09-16T11:46:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37965.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.37965.log.gz --verbose 2024-09-16T11:46:49.825 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.37726.log 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60765.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60765.log.gz 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37726.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.37726.log.gz --verbose 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.45631.log 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59801.log 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.45631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43227.log 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45631.log.gz 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52234.log 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59801.log.gz 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.43227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43227.log.gz 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33060.log 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.52234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45434.log 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52234.log.gz 2024-09-16T11:46:49.826 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33060.log.gz 2024-09-16T11:46:49.827 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38262.log 2024-09-16T11:46:49.827 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.45434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41771.log 2024-09-16T11:46:49.827 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45434.log.gz 2024-09-16T11:46:49.827 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38262.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38262.log.gz 2024-09-16T11:46:49.827 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61565.log 2024-09-16T11:46:49.827 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.41771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36937.log 2024-09-16T11:46:49.827 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41771.log.gz 2024-09-16T11:46:49.827 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61565.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.61565.log.gz 2024-09-16T11:46:49.827 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45817.log 2024-09-16T11:46:49.827 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.36937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39053.log 2024-09-16T11:46:49.827 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36937.log.gz 2024-09-16T11:46:49.827 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.45817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45817.log.gz 2024-09-16T11:46:49.828 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62195.log 2024-09-16T11:46:49.828 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38304.log 2024-09-16T11:46:49.828 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39053.log.gz 2024-09-16T11:46:49.828 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.62195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62195.log.gz 2024-09-16T11:46:49.828 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45252.log 2024-09-16T11:46:49.828 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60360.log 2024-09-16T11:46:49.828 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38304.log.gz 2024-09-16T11:46:49.828 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.45252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45252.log.gz 2024-09-16T11:46:49.828 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54858.log 2024-09-16T11:46:49.828 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38474.log 2024-09-16T11:46:49.828 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60360.log.gz 2024-09-16T11:46:49.828 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.54858.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54858.log.gz -5 2024-09-16T11:46:49.829 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46733.log 2024-09-16T11:46:49.829 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43732.log 2024-09-16T11:46:49.829 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38474.log.gz 2024-09-16T11:46:49.829 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.46733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46733.log.gz 2024-09-16T11:46:49.829 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53337.log 2024-09-16T11:46:49.829 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.43732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46352.log 2024-09-16T11:46:49.829 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43732.log.gz 2024-09-16T11:46:49.829 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.53337.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53337.log.gz 2024-09-16T11:46:49.829 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61775.log 2024-09-16T11:46:49.829 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.46352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37610.log 2024-09-16T11:46:49.829 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46352.log.gz 2024-09-16T11:46:49.829 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61775.log.gz 2024-09-16T11:46:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47239.log 2024-09-16T11:46:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37610.log.gz 2024-09-16T11:46:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45115.log 2024-09-16T11:46:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.47239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46036.log 2024-09-16T11:46:49.830 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47239.log.gz 2024-09-16T11:46:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.45115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49699.log 2024-09-16T11:46:49.830 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45115.log.gz 2024-09-16T11:46:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.46036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46036.log.gz 2024-09-16T11:46:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.52473.log 2024-09-16T11:46:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.49699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37721.log 2024-09-16T11:46:49.830 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49699.log.gz 2024-09-16T11:46:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/objectstore_tool.52473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38064.log 2024-09-16T11:46:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37721.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37721.log.gz 2024-09-16T11:46:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59885.log 2024-09-16T11:46:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47198.log 2024-09-16T11:46:49.831 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38064.log.gz 2024-09-16T11:46:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59885.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59885.log.gz 2024-09-16T11:46:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40169.log 2024-09-16T11:46:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51772.log 2024-09-16T11:46:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.47198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47198.log.gz 2024-09-16T11:46:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.40169.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40169.log.gz 2024-09-16T11:46:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40733.log 2024-09-16T11:46:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61523.log 2024-09-16T11:46:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.51772.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51772.log.gz 2024-09-16T11:46:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.40733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40733.log.gz 2024-09-16T11:46:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59381.log 2024-09-16T11:46:49.832 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44256.log 2024-09-16T11:46:49.832 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61523.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61523.log.gz 2024-09-16T11:46:49.832 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59381.log.gz 2024-09-16T11:46:49.832 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61859.log 2024-09-16T11:46:49.832 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60063.log 2024-09-16T11:46:49.832 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.44256.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44256.log.gz 2024-09-16T11:46:49.832 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40370.log 2024-09-16T11:46:49.832 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61859.log.gz 2024-09-16T11:46:49.832 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51284.log 2024-09-16T11:46:49.832 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60063.log.gz 2024-09-16T11:46:49.832 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.40370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40370.log.gz 2024-09-16T11:46:49.832 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37729.log 2024-09-16T11:46:49.832 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60935.log 2024-09-16T11:46:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.51284.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51284.log.gz 2024-09-16T11:46:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37729.log.gz 2024-09-16T11:46:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41589.log 2024-09-16T11:46:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61649.log 2024-09-16T11:46:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60935.log.gz 2024-09-16T11:46:49.833 INFO:teuthology.orchestra.run.smithi070.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.52473.log.gz 2024-09-16T11:46:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.41589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39428.log 2024-09-16T11:46:49.833 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41589.log.gz 2024-09-16T11:46:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.61649.log: /var/log/ceph/ceph-client.admin.40452.log 2024-09-16T11:46:49.833 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61649.log.gz 2024-09-16T11:46:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38600.log 2024-09-16T11:46:49.833 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39428.log.gz 2024-09-16T11:46:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.40452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40452.log.gz 2024-09-16T11:46:49.834 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44171.log 2024-09-16T11:46:49.834 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42392.log 2024-09-16T11:46:49.834 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38600.log.gz 2024-09-16T11:46:49.834 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39673.log 2024-09-16T11:46:49.834 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.44171.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44171.log.gz 2024-09-16T11:46:49.834 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.42392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62321.log 2024-09-16T11:46:49.834 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42392.log.gz 2024-09-16T11:46:49.834 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62403.log 2024-09-16T11:46:49.834 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39673.log.gz 2024-09-16T11:46:49.834 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.62321.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62321.log.gz 2024-09-16T11:46:49.834 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49317.log 2024-09-16T11:46:49.835 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.62403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51149.log 2024-09-16T11:46:49.835 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62403.log.gz 2024-09-16T11:46:49.835 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.49317.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49317.log.gz 2024-09-16T11:46:49.835 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37926.log 2024-09-16T11:46:49.835 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.51149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49149.log 2024-09-16T11:46:49.835 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51149.log.gz 2024-09-16T11:46:49.835 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37926.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37926.log.gz 2024-09-16T11:46:49.835 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T11:46:49.835 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.49149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38090.log 2024-09-16T11:46:49.835 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49149.log.gz 2024-09-16T11:46:49.835 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47155.log 2024-09-16T11:46:49.835 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38090.log.gz 2024-09-16T11:46:49.835 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61691.log 2024-09-16T11:46:49.836 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60984.log 2024-09-16T11:46:49.836 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.47155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47155.log.gz 2024-09-16T11:46:49.836 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61691.log.gz 2024-09-16T11:46:49.836 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44637.log 2024-09-16T11:46:49.836 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41989.log 2024-09-16T11:46:49.836 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60984.log.gz 2024-09-16T11:46:49.836 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.44637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40554.log 2024-09-16T11:46:49.836 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44637.log.gz 2024-09-16T11:46:49.836 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.41989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55105.log 2024-09-16T11:46:49.836 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41989.log.gz 2024-09-16T11:46:49.836 INFO:teuthology.orchestra.run.smithi070.stderr: 93.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T11:46:49.836 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.40554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44936.log 2024-09-16T11:46:49.837 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40554.log.gz 2024-09-16T11:46:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.55105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42968.log 2024-09-16T11:46:49.837 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55105.log.gz 2024-09-16T11:46:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.44936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44936.log.gz 2024-09-16T11:46:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38810.log 2024-09-16T11:46:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.42968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48912.log 2024-09-16T11:46:49.837 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42968.log.gz 2024-09-16T11:46:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38810.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38810.log.gz 2024-09-16T11:46:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59171.log 2024-09-16T11:46:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.48912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62027.log 2024-09-16T11:46:49.837 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48912.log.gz 2024-09-16T11:46:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59171.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.59171.log.gz 2024-09-16T11:46:49.838 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60985.log 2024-09-16T11:46:49.838 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.62027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44108.log 2024-09-16T11:46:49.838 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62027.log.gz 2024-09-16T11:46:49.838 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60985.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60985.log.gz 2024-09-16T11:46:49.838 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43170.log 2024-09-16T11:46:49.838 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.44108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61481.log 2024-09-16T11:46:49.838 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44108.log.gz 2024-09-16T11:46:49.838 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.43170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43170.log.gz 2024-09-16T11:46:49.838 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38432.log 2024-09-16T11:46:49.838 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61035.log 2024-09-16T11:46:49.839 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61481.log.gz 2024-09-16T11:46:49.839 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46616.log 2024-09-16T11:46:49.839 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38432.log.gz 2024-09-16T11:46:49.839 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60766.log 2024-09-16T11:46:49.839 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61035.log.gz 2024-09-16T11:46:49.839 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.46616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45511.log 2024-09-16T11:46:49.839 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46616.log.gz 2024-09-16T11:46:49.839 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45736.log 2024-09-16T11:46:49.839 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60766.log.gz 2024-09-16T11:46:49.839 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.45511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60359.log 2024-09-16T11:46:49.839 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45511.log.gz 2024-09-16T11:46:49.839 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.45736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59255.log 2024-09-16T11:46:49.840 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45736.log.gz 2024-09-16T11:46:49.840 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49618.log 2024-09-16T11:46:49.840 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60359.log.gz 2024-09-16T11:46:49.840 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38726.log 2024-09-16T11:46:49.840 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59255.log.gz 2024-09-16T11:46:49.840 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.49618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38063.log 2024-09-16T11:46:49.840 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49618.log.gz 2024-09-16T11:46:49.840 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33556.log 2024-09-16T11:46:49.840 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38726.log.gz 2024-09-16T11:46:49.840 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43412.log 2024-09-16T11:46:49.840 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38063.log.gz 2024-09-16T11:46:49.840 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60698.log 2024-09-16T11:46:49.841 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33556.log.gz 2024-09-16T11:46:49.841 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.43412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60066.log 2024-09-16T11:46:49.841 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43412.log.gz 2024-09-16T11:46:49.841 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61095.log 2024-09-16T11:46:49.841 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60698.log.gz 2024-09-16T11:46:49.841 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60066.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.60066.log.gz -- 2024-09-16T11:46:49.841 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.50178.log 2024-09-16T11:46:49.841 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61095.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-16T11:46:49.841 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61095.log.gz 2024-09-16T11:46:49.841 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.50178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37966.log 2024-09-16T11:46:49.841 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50178.log.gz 2024-09-16T11:46:49.841 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47439.log 2024-09-16T11:46:49.841 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37966.log.gz 2024-09-16T11:46:49.842 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38768.log 2024-09-16T11:46:49.842 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60639.log 2024-09-16T11:46:49.842 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.47439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47439.log.gz 2024-09-16T11:46:49.842 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61607.log 2024-09-16T11:46:49.842 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38768.log.gz 2024-09-16T11:46:49.842 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37730.log 2024-09-16T11:46:49.842 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60639.log.gz 2024-09-16T11:46:49.842 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38388.log 2024-09-16T11:46:49.842 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61607.log.gz 2024-09-16T11:46:49.842 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37212.log 2024-09-16T11:46:49.842 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37730.log.gz 2024-09-16T11:46:49.842 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38388.log.gz 2024-09-16T11:46:49.843 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39047.log 2024-09-16T11:46:49.843 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37390.log 2024-09-16T11:46:49.843 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37212.log.gz 2024-09-16T11:46:49.843 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42688.log 2024-09-16T11:46:49.843 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39047.log.gz 2024-09-16T11:46:49.843 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37390.log.gz 2024-09-16T11:46:49.843 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38065.log 2024-09-16T11:46:49.843 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62363.log 2024-09-16T11:46:49.843 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.42688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42688.log.gz 2024-09-16T11:46:49.843 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61817.log 2024-09-16T11:46:49.843 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38065.log.gz 2024-09-16T11:46:49.843 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.62363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62363.log.gz 2024-09-16T11:46:49.843 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50477.log 2024-09-16T11:46:49.844 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44420.log 2024-09-16T11:46:49.844 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61817.log.gz 2024-09-16T11:46:49.844 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.50477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42351.log 2024-09-16T11:46:49.844 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50477.log.gz 2024-09-16T11:46:49.844 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.44420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44420.log.gz 2024-09-16T11:46:49.844 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-16T11:46:49.844 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40548.log 2024-09-16T11:46:49.844 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.42351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42351.log.gz 2024-09-16T11:46:49.844 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43288.log 2024-09-16T11:46:49.844 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.40548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62237.log 2024-09-16T11:46:49.844 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40548.log.gz 2024-09-16T11:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.43288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59759.log 2024-09-16T11:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43288.log.gz 2024-09-16T11:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.62237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48257.log 2024-09-16T11:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62237.log.gz 2024-09-16T11:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59759.log: gzip -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59759.log.gz 2024-09-16T11:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.38684.log 2024-09-16T11:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.48257.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48257.log.gz 2024-09-16T11:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60064.log 2024-09-16T11:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55126.log 2024-09-16T11:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38684.log.gz 2024-09-16T11:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60064.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60064.log.gz 2024-09-16T11:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.50673.log 2024-09-16T11:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.55126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55126.log.gz 2024-09-16T11:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42690.log 2024-09-16T11:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.50673.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50673.log.gz 2024-09-16T11:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39912.log 2024-09-16T11:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.42690.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42690.log.gz 2024-09-16T11:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41532.log 2024-09-16T11:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.39912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43493.log 2024-09-16T11:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39912.log.gz 2024-09-16T11:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.41532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59633.log 2024-09-16T11:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41532.log.gz 2024-09-16T11:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.43493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62279.log 2024-09-16T11:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43493.log.gz 2024-09-16T11:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59717.log 2024-09-16T11:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59633.log.gz 2024-09-16T11:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.62279.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-16T11:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62279.log.gz 2024-09-16T11:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61397.log 2024-09-16T11:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59717.log.gz 2024-09-16T11:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47096.log 2024-09-16T11:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61397.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61397.log.gz 2024-09-16T11:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47796.log 2024-09-16T11:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.47096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47096.log.gz 2024-09-16T11:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61439.log 2024-09-16T11:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.47796.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47796.log.gz 2024-09-16T11:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60057.log 2024-09-16T11:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61439.log.gz 2024-09-16T11:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50950.log 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60057.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60057.log.gz 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37008.log 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.50950.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50950.log.gz 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51011.log 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37008.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.60799.log 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37008.log.gz 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.51011.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51011.log.gz 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5/var/log/ceph/ceph-client.admin.60799.log: --verbose -- /var/log/ceph/ceph-client.admin.60062.log 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60799.log.gz 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43289.log 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60062.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.60062.log.gz 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49275.log 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61943.log 2024-09-16T11:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.43289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43289.log.gz 2024-09-16T11:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.49275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49275.log.gz 2024-09-16T11:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49235.log 2024-09-16T11:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61943.log.gz 2024-09-16T11:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60301.log 2024-09-16T11:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.49235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49235.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61094.log 2024-09-16T11:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-09-16T11:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T11:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60301.log: /var/log/ceph/ceph-client.admin.61094.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61094.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60301.log.gz 2024-09-16T11:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-09-16T11:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59675.log 2024-09-16T11:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.53511.log 2024-09-16T11:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59675.log.gz 2024-09-16T11:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr: 90.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49959.log 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/objectstore_tool.53511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46775.log 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.49959.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49959.log.gz 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.53511.log.gz 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59297.log 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.46775.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59124.log 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46775.log.gz 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59843.log 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59297.log: /var/log/ceph/ceph-client.admin.59124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59297.log.gz 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59124.log.gz 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61141.log 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59843.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60300.log 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59843.log.gz 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40268.log 2024-09-16T11:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61141.log: /var/log/ceph/ceph-client.admin.60300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61141.log.gz 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60300.log.gz 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48713.log 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.40268.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37606.log 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40268.log.gz 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59591.log 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.48713.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48713.log.gz 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37606.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37606.log.gz 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38516.log 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59591.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62069.log 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59591.log.gz 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53417.log 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38516.log: /var/log/ceph/ceph-client.admin.62069.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38516.log.gz 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62069.log.gz 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38852.log 2024-09-16T11:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.53417.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47460.log 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53417.log.gz 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48994.log 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38852.log: /var/log/ceph/ceph-client.admin.47460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38852.log.gz 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47460.log.gz 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38642.log 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.48994.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44107.log 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48994.log.gz 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44031.log 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38642.log: /var/log/ceph/ceph-client.admin.44107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38642.log.gz 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44107.log.gz 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46372.log 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.44031.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37384.log 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44031.log.gz 2024-09-16T11:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42831.log 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.46372.log: /var/log/ceph/ceph-client.admin.37384.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46372.log.gz 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37384.log.gz 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60293.log 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.42831.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.42831.log.gz 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42689.log 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48075.log 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.42689.log: /var/log/ceph/ceph-client.admin.60293.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60293.log.gz 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42689.log.gz 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38062.log 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.48075.log: 0.0%gzip -5 --verbose -- /var/log/ceph/objectstore_tool.51102.log 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48075.log.gz 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60065.log 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38062.log.gz 2024-09-16T11:46:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/objectstore_tool.51102.log: 95.1% -- replaced with /var/log/ceph/objectstore_tool.51102.log.gz 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37128.log 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61313.log 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60065.log.gz 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:gzip/var/log/ceph/ceph-client.admin.37128.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44678.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37128.log.gz 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61034.log 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.44678.log: /var/log/ceph/ceph-client.admin.61313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44678.log.gz 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61313.log.gz 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41253.log 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61034.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48374.log 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61034.log.gz 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37170.log 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.41253.log: /var/log/ceph/ceph-client.admin.48374.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48374.log.gz 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41253.log.gz 2024-09-16T11:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50315.log 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37170.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61355.log 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37170.log.gz 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.54274.log 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.50315.log: /var/log/ceph/ceph-client.admin.61355.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50315.log.gz 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61355.log.gz 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38914.log 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/objectstore_tool.54274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60699.log 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.38914.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38914.log.gz 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.54274.log.gz 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool..log 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50039.log 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/objectstore_tool..log: /var/log/ceph/ceph-client.admin.60699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60699.log.gz 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41313.log 2024-09-16T11:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.50039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50039.log.gz 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47978.log 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.41313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41313.log.gz 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42113.log 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.47978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47978.log.gz 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59339.log 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.42113.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42113.log.gz 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59423.log 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59339.log.gz 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37070.log 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59423.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59423.log.gz 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37724.log 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37070.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37070.log.gz 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37389.log 2024-09-16T11:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37724.log.gz 2024-09-16T11:46:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40777.log 2024-09-16T11:46:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37389.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37389.log.gz 2024-09-16T11:46:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45976.log 2024-09-16T11:46:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.40777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40777.log.gz 2024-09-16T11:46:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-16T11:46:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.45976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45976.log.gz 2024-09-16T11:46:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60638.log 2024-09-16T11:46:49.868 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40129.log 2024-09-16T11:46:49.868 INFO:teuthology.orchestra.run.smithi070.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-16T11:46:49.868 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60638.log.gz 2024-09-16T11:46:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41115.log 2024-09-16T11:46:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.40129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40129.log.gz 2024-09-16T11:46:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45632.log 2024-09-16T11:46:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.41115.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41115.log.gz 2024-09-16T11:46:49.869 INFO:teuthology.orchestra.run.smithi070.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool..log.gz 2024-09-16T11:46:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61985.log 2024-09-16T11:46:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61733.log 2024-09-16T11:46:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.45632.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45632.log.gz 2024-09-16T11:46:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.61985.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61985.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59213.log 2024-09-16T11:46:49.869 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-09-16T11:46:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62153.log 2024-09-16T11:46:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59213.log: /var/log/ceph/ceph-client.admin.61733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61733.log.gz 2024-09-16T11:46:49.870 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59213.log.gz 2024-09-16T11:46:49.870 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44338.log 2024-09-16T11:46:49.870 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39790.log 2024-09-16T11:46:49.870 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.62153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62153.log.gz 2024-09-16T11:46:49.870 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.44338.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44338.log.gz 2024-09-16T11:46:49.870 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41871.log 2024-09-16T11:46:49.870 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59549.log 2024-09-16T11:46:49.870 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.41871.log: /var/log/ceph/ceph-client.admin.39790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39790.log.gz 2024-09-16T11:46:49.870 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41871.log.gz 2024-09-16T11:46:49.870 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60060.log 2024-09-16T11:46:49.870 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47679.log 2024-09-16T11:46:49.870 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.59549.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59549.log.gz 2024-09-16T11:46:49.870 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60060.log.gz 2024-09-16T11:46:49.871 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60542.log 2024-09-16T11:46:49.871 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43692.log 2024-09-16T11:46:49.871 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.47679.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47679.log.gz 2024-09-16T11:46:49.871 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.60542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60542.log.gz 2024-09-16T11:46:49.871 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37393.log 2024-09-16T11:46:49.871 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50756.log/var/log/ceph/ceph-client.admin.43692.log: 2024-09-16T11:46:49.871 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43692.log.gz 2024-09-16T11:46:49.871 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.37393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52111.log 2024-09-16T11:46:49.871 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37393.log.gz 2024-09-16T11:46:49.871 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.50756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44213.log 2024-09-16T11:46:49.871 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50756.log.gz/var/log/ceph/ceph-client.admin.52111.log: 2024-09-16T11:46:49.871 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52111.log.gz 2024-09-16T11:46:49.871 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.44213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44213.log.gz 2024-09-16T11:46:49.883 INFO:teuthology.orchestra.run.smithi192.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-16T11:46:49.930 INFO:teuthology.orchestra.run.smithi070.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-16T11:46:49.989 INFO:teuthology.orchestra.run.smithi070.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-16T11:46:50.167 INFO:teuthology.orchestra.run.smithi192.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-16T11:46:50.264 INFO:teuthology.orchestra.run.smithi070.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-16T11:46:50.628 INFO:teuthology.orchestra.run.smithi192.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-16T11:46:50.996 INFO:teuthology.orchestra.run.smithi192.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-16T11:46:51.800 INFO:teuthology.orchestra.run.smithi070.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-16T11:46:51.887 INFO:teuthology.orchestra.run.smithi070.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-16T11:46:52.140 INFO:teuthology.orchestra.run.smithi070.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-16T11:46:52.142 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-09-16T11:46:52.142 INFO:teuthology.orchestra.run.smithi070.stderr:real 0m2.447s 2024-09-16T11:46:52.142 INFO:teuthology.orchestra.run.smithi070.stderr:user 0m7.421s 2024-09-16T11:46:52.142 INFO:teuthology.orchestra.run.smithi070.stderr:sys 0m0.432s 2024-09-16T11:46:52.185 INFO:teuthology.orchestra.run.smithi192.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-16T11:46:52.187 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-09-16T11:46:52.187 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m2.456s 2024-09-16T11:46:52.187 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m4.936s 2024-09-16T11:46:52.187 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m0.236s 2024-09-16T11:46:52.188 INFO:tasks.ceph:Archiving logs... 2024-09-16T11:46:52.188 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907206/remote/smithi070/log 2024-09-16T11:46:52.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T11:46:52.428 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-16T11:46:53.329 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907206/remote/smithi192/log 2024-09-16T11:46:53.330 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T11:46:54.070 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T11:46:54.079 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T11:46:54.080 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T11:46:54.084 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T11:46:54.174 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T11:46:54.185 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T11:46:54.186 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T11:46:54.188 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T11:46:54.202 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-16T11:46:54.205 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-09-16T11:46:54.206 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T11:46:54.206 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-16T11:46:54.207 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 -460us[ -460us] +/- 38ms 2024-09-16T11:46:54.207 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 +1373us[+1374us] +/- 64ms 2024-09-16T11:46:54.207 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 -658us[ -657us] +/- 66ms 2024-09-16T11:46:54.207 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T11:46:54.210 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T11:46:54.210 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-09-16T11:46:54.210 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 -458us[ -468us] +/- 38ms 2024-09-16T11:46:54.210 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 +1380us[+1370us] +/- 64ms 2024-09-16T11:46:54.210 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 -643us[ -652us] +/- 66ms 2024-09-16T11:46:54.210 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T11:46:54.211 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T11:46:54.222 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T11:46:54.222 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T11:46:54.248 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T11:46:54.327 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T11:46:54.403 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T11:46:54.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:46:54.434 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T11:46:54.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:46:54.464 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T11:46:54.474 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T11:46:54.554 INFO:teuthology.task.internal:Duration was 631.171859 seconds 2024-09-16T11:46:54.555 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T11:46:54.623 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T11:46:54.624 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T11:46:54.628 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T11:46:54.666 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T11:46:54.668 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T11:46:54.947 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T11:46:54.947 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T11:46:54.948 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T11:46:54.968 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-09-16T11:46:54.968 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T11:46:55.024 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T11:46:55.024 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T11:46:55.026 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T11:46:55.093 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T11:46:55.093 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T11:46:55.096 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T11:46:55.245 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T11:46:55.254 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T11:46:55.254 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T11:46:55.279 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T11:46:55.304 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T11:46:55.315 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T11:46:55.321 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T11:46:55.342 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2024-09-16T11:46:55.367 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-09-16T11:46:55.416 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T11:46:55.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:46:55.431 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T11:46:55.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:46:55.471 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T11:46:55.481 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T11:46:55.482 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907206/remote/smithi070 2024-09-16T11:46:55.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T11:46:55.523 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907206/remote/smithi192 2024-09-16T11:46:55.523 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T11:46:55.555 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T11:46:55.556 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T11:46:55.558 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T11:46:55.606 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T11:46:55.617 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T11:46:55.618 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T11:46:55.639 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T11:46:55.639 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T11:46:55.641 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T11:46:55.654 INFO:teuthology.orchestra.run.smithi070.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 11:46 /home/ubuntu/cephtest 2024-09-16T11:46:55.661 INFO:teuthology.orchestra.run.smithi192.stdout: 265628 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 11:46 /home/ubuntu/cephtest 2024-09-16T11:46:55.662 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T11:46:55.673 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T11:46:55.741 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/cache thrashosds-health workloads/rbd_fsx_rate_limit} duration: 631.171858549118 flavor: default owner: scheduled_sangadi@teuthology success: true 2024-09-16T11:46:55.742 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T11:46:55.816 INFO:teuthology.run:pass